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2003 


---Upon commencing at 10:03 a.m. 

THE REGISTRAR: This hearing is again in 
session. Please be seated. 

THE CHAIRMAN: Mr. Shepherd. 

MR. SHEPHERD: Good morning, Mr. 
Chairman. 

Oneohe Matrer OLEyCimingy Lvdo anticipate 
being finished today, hopefully, sometime 
mid-afternoon. 

MITCHELL PIERSON ROTHMAN, 


PAUL JONATHAN BURKE, 
LILY BUJA-BIJUNAS; Resumed 


CROSS-EXAMINATION BY MR. SHEPHERD: (Cont'd) 


Or We spent some time on Thursday 
afternoon being as esoteric as we could possibly be on 
forecasting in general. I would like to deal with one 
other avast Am I echoing a lot here or is that my 
imagination? 

FROM THE FLOOR: Yes. 
==-Off \thesrecérd'discussion: 

MR. SHEPHERD: Is that better? 

FROM THE FLOOR: Yes. 

MR. SHEPHERD: Q. I would like to turn 
to the other area of general issues pelating to 
forecasting and talk about practical aspects affecting 


how you forecast, that is, non-professional aspects. 
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Rothman, Burke,Buja-Bijunas 2004 
cr ex (Shepherd) 


Mr. Rothman, I wonder if you can turn to 
exhibit -- is this 128? The excerpt from your article, 
is that 128? And before I go into this, let me say at 
the outset, Mr. Rothman, that there is no suggestion in 
this series of questions that any of you would do 
anything that is unprofessional or unethical or 
anything like that. That is not the point of the 
series of questions. 

So, looking at Exhibit 128, Mr. Rothman, 
this is, in fact, an excerpt from an article you wrote 
about 12 years ago; is that correct? 

MR. ROTHMAN: A. It was published as an 
article but it was actually written as a speech as you 
can tell from the language and informality of the 
language. 

Ouse Yes,,OkaysdoAnd at isgabouted26years 
ago. This is before you were at Ontario Hydro? 

A. I infer from evidence within the full 
article that it was written at least in 1978. It may 
have been published a year or so later. 

And yes, I was with the Canadian Imperial 
Bank of Commerce at the time. 

Q. Now, at the top of page 362 there, 
the only page that is in the exhibit, you say, talking 


here about forecasting in the intermediate term, and 
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Rothman, Burke,Buja-Bijunas 2005 


cr ex (Shepherd) 


this article has nothing to do with load®forecasting, 
does it? 

A. No, it has to do with forecasting 
energy investment, and energy investment is a fraction 
of GDP. 

Q. Starting the third line down, you 
say, "First, it is a relatively safe exercise..." - 
there you are referring to forecasting in the 
intermediate term - "...because no one is likely to 
call up in 12 years to point how wrong you were." That 
sentiment presumably is at least as true with long-term 
forecasts. 

A. Well, you are calling me up in 12 
years to see how wrong I was. (Laughter) 

Q. Not about these forecasts. 

A. And in fact, I looked it up. We 
weren't that wrong. So... (Laughter) 

Yes, that was partly meant as a -- as I 
Said, it was an informal speech rather than a paper. 
But what I really meant to say there was that with this 
length of forecast, either in this context or in the 
context of load forecasts, you really don't know how 
accurate the forecast was until quite a bit after the 
forecast was made. And that is a problem that we have 


been dealing with really throughout much of these 
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Rothman, Burke,Buja-Bijunas 2006 


cr ex (Shepherd) 


hearings. 

It has really been underlying a lot of 
what the discussion has been here about uncertainty, 
and that essentially it is a recognition that the 
forecasts have to be uncertain and that we have to be 
able to express that uncertainty somehow, because we 
won't know until for many years whether the forecast 
that we have made now is accurate. 

It leads to a question that we have been 
talking about, again, through much of these hearings. 
On what kind of evidence do you change a forecast that 
hasn't yet been proven incorrect? We change our 
forecasts for the year 2000 to 2005 annually. How do 
we know that the original forecasts, the forecast that 
we are changing is worse than the new one that we are 
making. 

And again as we selene: the record, in 
1981 and '82, Ontario Hydro did change forecasts for 
the year 2000 and made them worse by overreacting to 
what was then a cyclical downturn. 

Q. Mr. Rothman, this sort of practical 
aspect, the fact that 25 years from now you probably 
won't be working for Ontario Hydro - I am just 
guessing - that doesn't mean you would necessarily take 


some sort of devil-may-care attitude to forecasting; 
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Rothman, Burke,Buja-Bijunas 2007 


cr ex (Shepherd) 


does it? 

A NOGhat) ates Siiisatd; this was a 
speech and intended to be relatively light-hearted way 
of expressing what I think is a series consideration: 
that you are making a forecast for some fairly long 
time in the future and you won't know how accurate it 
was until sometime later. 

As I said, I went back and I looked at 
this and what we were essentially trying to estimate 
was GDP as a fraction -- sorry, energy investment as a 
fraction of total GDP, and we were within about a 
percent except for the '86 to '90 period -- 

Q. Mr. Rothman, I am not trying to get 
into that. I am not questioning that your forecast 
then was right. It might have been dead-on and I am 
pextectlyshappyawithythats Mam ejust-trying to'get at 
the practical factors affecting forecasting. 

Would it be fair to say that because of 
the nature of long-term forecasting, that you are more 
dependent on your personal professional standards. 
That is, the standards you put into making forecasts to 
sort of focus and motivate the correctness of your 
forecast because nobody is going to come back to you in 
25 years? 


A. I am not quite sure what you mean by 
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Rothman, Burke,Buja-Bijunas 2008 
cr ex (Shepherd) 


that question. And I am not trying to be obstructive 
here. I am just trying to get at exactly what kind of 
question you are asking me. If you could rephrase it. 

Q. In many circumstances in anybody's 
job, the control factor, if you like, the Ching--that 
makes them more likely to do a good job is the fact 
that if they do a bad job, next year they are going to 
get whacked in compensation, in losing their job, in 
being criticized, whatever. That's not true in 
long-term forecasting, is it? 

And what I am getting at, isn't the 
result of that that you have to set personal 
professional standards for your work because nobody 
external is going to be able to say you were wrong? 

A. We do several things about how we set 
standards for the quality of the forecast and how we 
monitor that. First of all, obviously, there is some 
ability to monitor short-term forecast; and while we 
don't have any guarantee that an accurate short-term 
forecast will be accurate in the long term, again 
referring to the history of the past decade, it does 
give at least some comfort. 

Secondly, we do take some care to discuss 
these forecasts with outside experts, to discuss our 


methodology and to discuss our results. That is an 
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Rothman, Burke,Buja-Bijunas 2009 
cr ex (Shepherd) 


important check, I think, both on the quality of what 
we are doing and how well we are doing it. 

And finally, I agree with you that we try 
to maintain high professional standards. I think we do 
maintain high professional standards within the 
division and for our forecast. The feedback that we 
get from the outside community tells us that they agree 
with that statement that we do maintain high 
professional standards. We try also to maintain 
forecasting methodologies that are consistent with the 
best professional, best current professional standards. 

Q. Now that I guess brings us, your 
pe: eee Te the outside expert, brings us to the 
second part of that quote, where you said: 

"Second, it tends to produce a 
consensus because no forecaster wants to 
appear too different from the others 
because he or she will then have to 
justify the difference." 

Perhaps you can just expand on that 
briefly. 

A. Yes, that really was intended as more 
humorous than serious. I could mae 

Q. Sorry, Mr. Rothman, do you agree with 


these statements in this article that was published. 


Farr & Associates Reporting, Inc. 


Rothman, Burke,Buja-Bijunas 2010 


cr ex (Shepherd) 


This was published in a book; right? 

Ba. Des, 

Q. I don't have a copy of the speech 
here. This was published in a book on this subject. 
Now do you agree with the two statements that I have 
quoted or not? 

A. The book was a set of papers 
delivered at a conference. 

Q. The book wasn't a piece of humour? 

Ay. No. 

Q. Do you agree with the two statements 
I have quoted or not? 

A. I agree with the first statement, as 
Ix isaid: 

The second statement, I would put it as 
the weight of ,:probability -:sthatuisy = can thinkvo£t 
reasons why it would not produce a consensus and 
reasons why it would. So, I don't hold that view 


strongly. 
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Rothman, Burke,Buja-Bijunas 2011 
cr ex (Shepherd) 


(LO TSeaem-s 91°The* firstpyasebisaiad;*®1I think’ it isca 
light-hearted statement of what I see as an important 
aspect of long-term forecasting. 

The second is a light-hearted statement 
again of what I see as a less serious problem with 
long-term forecasting, and that's partly because there 
Simply isn't the large community here, and we do keep 
in touch with each other, but I don't think that we 
deliberately try to produce consensus. 

Q. Can you tell us what the Delphi 
method is, can you describe it briefly? 

A. It's an interactive process of 
arriving at consensus among forecasters. 

Q. How does it work? 

A. Typically, there is a first survey 
for some forecast variable done among a group of 
experts of forecasters.@! Thetresultseoft*that first 
Survey are then fed back to each of the participants 
and they are asked to produce new forecasts, bearing in 
mind what everyone else has produced, until eventually 
a consensus is reached and that process is repeated 
until eventually a consensus is reached. 

Q. You don't have to actually get toa 
consensus to use Delphi, right? You can also use it to 


produce a probability distribution? 
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Rothman, Burke,Buja-Bijunas 2012 
cr ex (Shepherd) 


A. Yes, you can use it to produce a 
range, that's right. And it is, as Mr. Burke has 
testified, used in that way by NAERC. 

Q. When you use techniques like Delphi 
-- and you have used those in the past, haven't you? 

A. I haven't. We haven't within Ontario 
Hydro. 

Q. You haven't? 

A. Not as far as I know. 

Q. We will come back to that then. 

These are just two of the practical 
issues that affect how you work. I am trying to get a 
sense of what sort of other practical issues, 
non-professional issues, affect the results of your -- 
could affect the results of your forecast. Can you 
give us some other examples of practical things that 
would limit or direct or put pressures on your 
forecasts? 

A. I don't know that this issue of 
consensus puts issues on a forecast. I think it is 
important, this point of consensus was to say that 
everyone is aware, there is a small community and 
people who are in it are aware of what other people are 
doing. I can recall, for example, a continuing set of 


conversations I had with a person who was then in 
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Rothman,Burke,Buja-Bijunas 2013 
cr ex (Shepherd) 


charge of forecasting for the Federal Department of 
Energy, Mines and Resources. His forecast of 
electricity sales growth for Ontario was much higher 
than Ontario Hydro's by about as much as 2 per cent per 
year, to the year 2000. And we had continuing 
discussions about why our forecasts were so different. 
And the next year his had gone down and ours had come 
up a little and I continued to say to him, we are going 
to converge but you are coming down faster than we are 
going up. 

It's that kind of awareness of other 
people's peeaee and of the reasons for that, that I 
think are important within this community, and we don't 
have to reach or look for consensus within that 
community, but it is important that we have some 
understanding within that community about weet the 
sources of differences are, and I think we do get that. 

Q. Mr. Rothman, I appreciate that you 
are saying that the consensus point is not a strong 
factor but perhaps we could get to the question I 
asked, which is, what other factors, what other 
practical factors impact on your forecasts? 

A. Well, the practical factors of 
forecasting in general will impact on any forecast. We 


have talked in this hearing a fair amount about data 
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Rothman, Burke,Buja-Bijunas 2014 
cr ex (Shepherd) 


problems, and you talked, I think it was you with whom 
I was talking about assumptions and how they feed 
through into a forecast. We said then that one of the 
things that may constrain the variables that you can 
choose as explanatory variables is availability of data 
Or information on them, availability of forecasts for 
them. That's quite a practical consideration of making 
a forecast. 

Q. Tell us some others. 

A. Paul, can you help me here? 

There are methodological considerations 
as we have talked about. 

Q. What about things like the lack of 
available resources, not having enough money or enough 
people to do the job that you might otherwise like to 
do, is that a practical limitation? 

A. Well, it's always a practical 
consideration. Resources are always a practical 
constraint, and I think we have had really, in one 
perspective, we have quite good resources. By another 
perspective we can always -- to do the job that we 
would like to do and we think it is important to do, we 
could use more. 

Q. Haven't both Dr. Buja-Bijunas and Mr. 


Burke testified a number of times that there were 
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Rothman, Burke,Buja-Bijunas 2015 
cr ex (Shepherd) 


things that they would have liked to better that they 
couldn't because they just didn't have the resources? 

A. Absolutely. 

Q. It's a common problem, right? 

A. Absolutely. But I think that there 
are many things that we could usefully do, that I think 
it would be important for us to do, but that within an 
organization like Ontario Hydro simply not -- we just 
simply haven't had the resources to do. 

I think that the organization has by 
creating the economics and forecast division, 
maintaining the resources to it that we can have ina 
time of resource constraints throughout the 
Organization, indicated a strong commitment to 
forecasting and to the quality -- and an understanding 
and acceptance of the quality of what we are doing. 

Q. But there is always something more 
you could do no matter how good a job you are doing, 
there is a always something more you could be doing if 
you had the resources. 

A. Sure. In fact, one of the problems 
that I started out talking about, which was data 
constraints, is not unrelated to the problem of 
resources. 


Q. And you, similarly, have time 
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Rothman,Burke,Buja-Bijunas 2016 
cr ex (Shepherd) 


pressures. Didn't you, Mr. Burke, testify that perhaps 
one of the reasons why you had the problem with the 
bandwidth that you talked about on Wednesday was the 
fact that the 1990 load forecast was done under pretty 
severe pressures of time and otherwise, and it's the 
sort of mistake that anybody can make? 

MR. BURKE: A. Well, that's correct, and 
I think the particular added factor this year was this 
hearing itself and the preparation for it. So, that's 
really what I was referring to in that instance. 

Q. It's a common problem, like in 
anybody's job, right, like preparing cross-examination 
for a hearing, sometimes you have time pressures and 
you, == 

A. One has to set one's priorities and 
do what one can do with the available resources. 

Q. Now, Mr. Rothman you were in the 
private ‘sector; forka; while. andvisn't 4t truee@that in 
some private sector firms the forecasters are subjected 
to sort of the corporate culture pressures, if you 
like, Organizational biases, expectations of the 
Organization that forecasts will go in a certain 
direction, whether or not they succumb to them there 
are those pressures there, right? 


MR. ROTHMAN: A. I wouldn't say that I 
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Rothman, Burke,Buja-Bijunas 2017 
cr ex (Shepherd) 


observed that at CIBC. I think that they were looking 
to the forecasters for, insofar as they used the 
forecast for internal purposes, I think they were 
looking for the best forecasts available. 

Q. So, have you not seen that sort of 
affect in other organizations, where the organization 
sort of has a way of thinking and the forecasters are 
expected to match that way of thinking in some way? 

A. It is observed, but from outside the 
Organization. That is people tend to say this 
Organization or that one tends to be an optimistic 
forecast or it tends to be a pessimistic forecast or it 
tends to forecast patterns that would be beneficial to 
that organization. But that's really armchair 
empiricism and unless you have been within the 
Organization you have seen to what extent the 
forecasters are independent, it's very hard to tell. 

Q. So outsiders will sometimes say that 
of forecasters but in your experience it's not normally 
true in fact; is that fair? 

A. I don't have broad experience. I 
talked to other forecasters in the community and I 
haven't heard them say that they have been pressured by 
their organizations to forecast in one direction or 


another, and the organizations that I have been in have 
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Rothman,Burke,Buja-Bijunas 2018 
cr ex (Shepherd) 


not influenced the direction of my forecasts but I 
can't say whether this armchair empiricism from outside 
the organization is correct or not. 

Q. Another area that is sometimes talked 
about is specific demands by management or other people 
within the organization for specific results in the 
forecast. You have talked about them yourself, the 
Criticism you have received internally about the size 
of your bandwidth and the pressure, if you like, to 
narrow the bandwidth. Again, I am not suggesting you 
did anything unprofessional, but certainly there was a 
motivation there caused by internal pressures, wasn't 
there? 

A. I got a very clear message from my 
boss at the time, Norm McConnell that he thought that 
the bandwidth methodology needed to be revisited, He 
Since I had expressed to him concerns of my own about 
the bandwidth methodology it was revisited. 

I would say that it was clear that he 
felt that the bandwidth was too wide to be as useful as 
the planner would like it to be. But I think it was 
also clear that if I were to revisit the methodology 
and to tell him and the organization that in the best 
professional opinion of myself and my staff, 


particularly Mr. Burke, that the bandwidth was as it 
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Rothman, Burke,Buja-Bijunas 2019 
cr ex (Shepherd) 


should be, then it would have stayed as it was. I am 
Firmly convinced of that. 

Q. So, this is sort of another example 
of you having to -- and I'm not suggesting in the face 
of disagreement, but you have to stick to your 
professional integrity, if you like, and do what you 
believe is right, regardless of what the organization 


would like to have? 
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Rothman, Burke,Buja-Bijunas 2020 
cr ex (Shepherd) 


[10:25 a.m.] A. I think the organization has every 
right to tell me what things it wants me to forecast, 
and I think that if the organization says, as it did in 
this case, we would like you to revisit that 
methodology, it has every right to do that. And I 
think we should responsive to such signals. 

I have not felt myself to be ina 
position where I have to defend the integrity of the 
forecast against an organization that wants to change 
1t.; I think: the organization has accepted and 
respected that the forecast and the forecasters have to 
maintain that integrity and haven't made -- haven't put 
pressure on to change the forecast. 

There is an interaction between the 
forecaster and the organization. We certainly have 
discussions about whether we will do an update to a 
forecast at a given time, especially in the context of 
the Ontario Energy Board, and I think those are quite 
legitimate. If we were to do an update now, what would 
happen, and we've had discussions of those kinds. 

We've had discussions with the 
Organization that we have kind of an informal agreement 
with the organization around the Ontario Energy Board 
that if we're going to make a forecast update, we won't 


make a change if it's less than a 1 per cent change in 
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Rothman, Burke,Buja-Bijunas 2021 
cr ex (Shepherd) 


the load forecast generally, because making a small 
change produces a lot of work, and we've got to go 
through their calculations again, and it's not 
necessarily worth it for a small change of 1 per cent. 

We have those kinds of conversations with 
people in the organization about what we're going to 
do, when we're going to do it, how we're going to do 
it, and I think those are all legitimate -- 

Q. But those aren't discussion about 
what your results should be? 

A. Those are not discussions about what 
the results should be. 

Q. Nobody ever comes to you and says, we 
want the median load forecast to be higher or lower. 

We don't think we like yours very much? 

BLeaNo’: 

MR. BURKE: A. Can I just add something 
to what Mr. Rothman said? 

Q. Sure. 

A. At the time that the initiation of 
the revisiting of the bandwidth was being discussed, 
there was external feedback to Ontario Hydro concerning 
the bandwidth itself. It has been examined by the 
Select Committee on Ontario Hydro and also the 


technical advisory panel, whose document you've 
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Rothman, Burke,Buja-Bijunas 2022 
cr ex (Shepherd) 


included in your exhibits. Both of them made the 
judgment - I don't think either of them did analysis of 
this question - but both of them made the judgment that 
they considered the band to be too wide for planning 
purposes. 

And that was a piece of feedback that we 
were given to consider. I think the Ministry of Energy 
has also expressed the same view. I can't remember 
exactly the document, but in sort of a formal way that 
for planning purposes the old bandwidth, the pre-1988 
bandwidth, was too wide for planning purposes. 

O% Well, you've anticipated me, Mr. 
Burke.re Coulds youssturm to Exhibit 68, page 20. That's 
the electricity planning technical advisory panel. All 
right? And their role was to review a predecessor to 
the plan analytically. 

And the technical advisory panel says at 
page 20: 

",...in Hydro's estimates there is a 40 
per cent probability that actual load 
growth will fall outside (it is not known 
how far outside) the range between high 
and low. Moreover, the range itself is 
very wide: The 26,000 MW difference 


between the high and low forecasts in the 
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year 2007 is equivalent to more than 30 

new Darlington size nuclear reactors! 

This adds up to an extraordinary amount 

of uncertainty confronting the 

decision-makers." 

Thates the criticism youre talking 
about,™sn iti t? 

A. That is the extent of it, yes. 

O.-weNow, “ish twis Pt itruekthatuthat was 
one a lot of criticisms that you had on various 
aspects, and that you've essentially changed your 
methodology in virtually every way that they suggested 
you should. They said, "Do more end-use forecasting." 
You're doing it. They said, "Upgrade and improve your 
models." And you've done it. Isn't that true that 
you've done virtually everything they asked you to do? 

A. Well, they aren't the only group that 
has asked for that sort of thing. The admonishment to 
upgrade end-use forecasting has been made since the 
late 70'sit I think’ the CEG put’ on the ‘table a 
compendium of recommendations over the years to do just 
that, to improve our econometric and end-use 
forecasting as almost the motherhood in the load 
forecasting business. 


So, I don't consider those comments about 
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"improve end-use, improve econometric", were 
particularly a departure from the trend we were already 
ON. 

The observation about the uncertainty 
band though came at a time when we were uSing one 
methodology and several other sources. I guess this 
particular group came after the Select Committee, that 
it also made the same comment. 

So, it was not something where we were on 
a path to already. It was something that required us 
to rethink what we were doing to see whether we still 
agreed with our methodology or not. And the 
methodology was developed I think in the late 70's or 
early 80's. 

MR. ROTHMAN: A. I don't know. 

MR. BURKE: A. And, I guess, one of the 
things that comes with having sort of new people in the 
division is that new ideas come forward. 

O..,/ And amenotwbeingyucriticale Mri: 
Burke. Quite the opposite. You had quite a lot of 
Criticism in the mid-80's that you responded to; isn't 
thac, trues 

A. We responded in the sense that Mri. 
Rothman has said. We revisited the analysis to see 


whether or not we could justify the range we had. 
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Q. But didn't -- 

MR. ROTHMAN: A. If I could just adda 
bit about this technical advisory panel. There were a 
few criticisms that they made which were essentially -- 
or a few recommendations that they made, which were 
essentially statements saying we should be doing what 
we're doing. 

I think they may or may not have 
understood that we were already doing it. But in some 
cases they said to do things that we were already 
doing. Improving end-use modelling, they certainly 
understood was something that we were already doing. 

I believe from informal conversations 
that it may have been their hope that we could 
excelerate or increase the level of effort on end-use 
modelling, at least partly in response to their 
recommendation. 

Q. I just get the impression, and maybe 
you Can; correct ‘me if (I'm wrong... I’ just get the 
impression that to a certain extent you're damned if 
you do and damned if you don't. In the 80's you had a 
certain set of things that you were doing, and you were 
changing them and improving them, partly as a result of 
criticism. And now you've done everything everybody 


asked you to do, and now we're all sitting here 
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Criticizing the result. Is that a fair conclusion? 

For example, you were told, "Narrow the 
uncertainly band. This is useless for planners." So 
you did. And now we're sitting here telling you your 
uncertainty band is too narrow. Isn't that right? 

A. I don't think I've heard anybody say 
that yet. 

QO.» OR.m  OKay:? 

A. And I haven't heard anybody saying 
that they don't think we should be doing end use 
analysis at the level that we've been doing it. We've 
had a lot of discussions here about how we've done that 
analysis and what its results have been, but I don't 
think I've heard anybody at these hearings say we 
shouldn't have been doing that end-use analysis or that 
they .don}t thinkewe sivalid- 

Well, we have, of course, had lots of 
discussions about the end-use analysis and some 
questions as to its date and methodology, but I don't 


think I've heard anybody say we shouldn't have done it. 
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[10:38 a.m.]) Of. The suggestion nas in Fact “been made; 
hasn't it, Mr. Rothman, that your uncertainty band 
narrowing was simply the result of internal pressures, 
not truly the result of your discovery of a new and 
better method, hasn't it? 

I refer you to your cross-examination by 
the Coalition over a week ago today. Wasn't that 
suggestion, in fact, made to you directly? 

MR. B. CAMPBELL: Could we have a 
Cranscriptereféréence? *1I* don"t*récall it being*@put ‘in 
those terms. 

. MR. SHEPHERD: Just a second. I'll see 
PETIT ACAnsElnadeie! 

Q. You don't recall such a suggestion 
being made to you? 

MR. B. CAMPBELL: That there was internal 
pressure to narrow it in the later end? 

MR. SHEPHERD: Yes. 

MR. “B+ ‘CAMPBELLS YP don’t "recall that. 

That is, to have it converge is what I 
mean by narrowing. 

THE CHAIRMAN: There is no doubt that 
there was some criticism about the width of the band 
communicated-- 


MR. B. CAMPBELL: I think that has been 
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spoken to this morning. 

THE CHAIRMAN: --and that's been spoken 
about this morning. 

Are you Saying there is something 
different in that -- 

MR. ROTHMAN: We made no bones, Mr. 
Shepherd, that the external criticism and the internal 
statements were that the bands were wider than was 
useful to the planners for planning purposes. 

So, if that's all you are getting at, 
yes, we have agreed with that, and been quite clear 
about it all along, that the message we were getting 
from both inside and outside the Organization is that 
the bands are too wide. 

But the instructions that I got from my 
boss were not 'Find a methodology that will produce a 
narrower band.' The instructions that I got were 'Use 
your professional methodologies to revisit the 
bandwidth methodology and decide whether you, stil. Like 
the one that you have or whether you have something 
else that you would prefer.' And his instructions were 
not 'Produce a narrower bandwidth.' His instructions 
were ‘Revisit the bandwidth methodology.' And he -- if 
you know Norm McConnell. 


MR .+SHEPHERDs.*0.» Indo. 
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MR. ROTHMAN: A. He used the words 'This 
is your professional judgment.' 

Q. Maybe I am just being dense, but I 
guess it's -- are you telling us that Mr. McConnell did 
not come to you or anybody did not come to you and say, 
"We have been criticised for having a bandwidth that is 
too wide. This is a problem. Will you look at it?' 

Did he just come to you and say, 'I don't 
know about this methodology. Why don't you see if you 
can find another one, wider or narrower, it doesn't 
matter.' 

THE CHAIRMAN: This may be too simple, 
but surely it's implicit that an improvement in 
methodology will narrow a bandwidth, otherwise why 
would you improve the methodology? 

MR. SHEPHERD + peMpeeChairman jbledon’t 
think Mr. Rothman was saying that he necessarily was 
asked to improve the methodology. 

Q. He was asked to revisit it and if 
necessary find another methodology; isn't that correct? 
Which presumably would be better. 


MR. ROTHMAN: A. First, Mr. Chairman, I 


think I am going to disagree slightly there. 


THE CHAIRMAN: You can disagree 


completely as far as I am concerned. (Laughter ) 
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MR. ROTHMAN: Certainly, it's possible 
that we could have a bandwidth that was unreasonably 
narrow, and so improving it would be widening it to a 
more reasonable width. So improving it means that we 
have a bandwidth that is one which we feel better 
represents! the’/trte’ uncertainty. That's the first 
criterion. 

The second criterion, we would hope, is 
one that is better, more useful for planning. And I 
was told effectively, I believe I was told by my boss 
and I think he was quite explicit, I think he said 'You 
Should revisit your bandwidth methodology and produce a 
new one with which you are, if you can, produce a new 
one with which you are professionally more comfortable, 
which you think better represents the true uncertainty 
bandwidth than does the present one.' 

MR. SHEPHERD: Q. Mr. McConnell wasn't 
asking you to produce a wider bandwidth, was he? 

MR. ROTHMAN: A. When I got the 
instructions, he didn't say anything about wider or 
narrower. I don't want to split hairs here, Mr. 
Shepherd, it's quite clear that that was in a context 
where both the system planning people and Mr. McConnell 
and the select committee and the technical advisory 


panel and others had criticized our bandwidth for being 
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COO. wide 4S0,-Vvou know, SLi doneaek nowexs 


at 


think we've 


got enough here. I think we know what the context was. 


And as I said, I had instructions to get 


a better bandwidth methodology. If I had come to Mr. 


McConnell and said, 'I can't do any better than the 


existing bandwidth methodology; that is the best 


methodology that we can produce because we think it 


best represents what the uncertainty band truly is,' I 


believe that that would have been accepted. 


I think he 


would have said 'That is your best professional 


judgment?' '‘'Yes.' ‘Then you have to go with your 


professional judgment. You are the professional.' I 


think he would have been disappointed, but he would 


have accepted that. And that's where we are. 


Q. Fair enough. 


A. Well, let's leave it at that. I 


think the other thing we might say, though — Jhiesaid 


let's leave it that and I am not going to. 


Well... 


MR. B. CAMPBELL: This may not be your 


choice. 


MR. SHEPHERD: I am having a nice rest 


here, so you can go ahead. 


MR. ROTHMAN: All I wanted to say was 


that I had been uncomfortable with the undertainty band 


methodology for some years and had looked at it briefly 
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in 1983 when I first got responsibility for the load 
forecast. 

But, quite frankly, I had neither the 
time nor as much technical expertise as Mr. Burke to 
carry through a thorough review, so it was not a new 
question in 1988. It was something that had really 
kind of been on the agenda for some time. 

And), *ifo fact 7s Ilethink that: I had actually 
said ithatsiswouldsdorrte inges7, and we finally got £6 
it in '88, so it was not a sudden thing that we 
reviewed the bandwidth methodology. 

MR. SHEPHERD: Q. Just before I get to 
your economic forecast. Just one other sort of minor 
thing really. Mr. Burke you have raised it, and Mr. 
Rothman, you have raised it as well, in the last couple 
of weeks, the question of scaling of charts. I just 
want to look at it briefly for a second. 

The graphic I am showing you, I believe, 
is your graphic of goods production as a percentage of 
total Ontario GDP? 

MR. ROTHMAN: A. It certainly look like 
1) a 

Owe Thyvs) ieevours, right? 

Am I right in understanding that you are 


projecting a continuation of the past trend away from 
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goods production but at a shallower rate than in the 
past? 

A. Yes. 

Q. Is that your testimony? 

Now you have provided us with the actual 
numbers behind this graph; haven't you? Through your 
counsel, did you not provide us with the numerical data 
behind this graph? 

A, Yes. 

Q. Everybody nods. 

Well, I would like to show you then a 
second overhead which is the same data but rescaled. 
Can you tell us what the difference between this chart 
and the one that you just saw, is? Numbers 1 and 2? 

Ae lean noc. 

MS. PATTERSON: What was the exhibit 
number of the original? 

MR. SHEPHERD: The original one, I 
believe, was Exhibit 100, overhead No. 6 -- sorry, yes, 
it is now 126, page l. 

THE CHAIRMAN: But what was the original 
one though? 

MR. SHEPHERD: It was originally 100, 
page 6, I believe. Or 99 page 6. 


MR. ROTHMAN: 99. 
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MR. SHEPHERD: Was it 99, page 6? Okay. 

THE CHAIRMAN: And yours is 126... 

MR. SHEPHERD: 126, page 1. And the one 
that is up now is page 2. 

THE CHAIRMAN: Page 2. 

MR. ROTHMAN: I am not sure what you mean 
by the difference. One is on an expanded vertical 
scale. 

MR. SHEPHERD: Q. I guess I am going to 
ask you. Why did you select the scale you did for your 
exhibit? It looks sort of strange to me that all the 
numbers are sorts of pushed into the centre of the 
chart. 

MR. ROTHMAN: A. Well, I must admit that 
I have again violated one of my -- one of Mr. 
McConnell's principles and did not zero-base this 
chart. He was a real stickler for saying 'If you want 
to know what's really happening with the data, then you 
have to give us a chart that has zero as the basis.' 

If I had done that it would have been even more -- 

Q. It would have been flatter; right? 

A. But I am not sure that I chose the 
scale for this chart. It may just have been done by 
the person whom I asked to plot it. 


But I thought that that scale was 
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reasonable to indicate the pattern that I wanted. And 
when you are looking at long-term patterns like that, 
it is preferable not to have as a blown up a scale as 
you used for this chart in general. This makes it look 
as if we have quite a volatile series. And, in fact, 
if you look at the previous chart or took it in as a 
zero-base’ chart; it isn't all®that» volatile a»series. 
What this does is emphasize the cyclical variations to 
some extent. 

Q. Doesn't it also tend to demonstrate 
that in fact the trend that you are projecting is 
virtually identical to the trend of the previous 30 
years? 

A. Well, it depends. 

©. altigustideoksyvisuallyrtormet="1 
don't know. 

A. It depends on how you read it. If I 
were to take a pencil - and I don't know how this is 
ever going to get into a transcript - but if I were to 
take a pencil and look for a trend in this part of it, 
you could easily have a trend like that. And the one 
that you've got in the forecast is like that. 

Q. And that would be from 1981 into the 
Future; wouldn't it? 


A. I think in fact that,’ you know, you 
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can -- on the other hand, as you are suggesting, you 
can take a trend that looks like that and then one that 
looks like that and they look pretty parallel. So, it 
is all a question of how you read it. In fact, as I 
recall, what I looked at, I looked at it in terms of 
kind of a per cent per year in change. 

Oe Yesr 

A. And as I look at it as a per cent per 
year change, the forecast rate - I think, my numbers 
are right off the top of my head - is something like 
half that of the historical period. 

Q. The rate of change is half what it 
was in the past? 

A. Well, yes, I mean... We can look it 
up. 

Q. Well, perhaps the easiest way rather 
Perl iss 

A. Exhibit 15 has the data. 

Q. Excuse me, Mr. Rothman. Perhaps the 
easiest way to deal with this, if your counsel has no 
objection, is I wonder if you could undertake to 
provide us with the numerical data or a reference to it 
so that we can look at it. 

AJ 45's! frledo thes iBznebit 257 ‘the 


tables at the back. 
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Q. Do you have the comparative rate of 
change for the last 30 years and the next 25? 

THE CHAIRMAN: Hold on a second. We will 
just take one thing at a time. Let's make sure we have 
got the right table. 

MR. ROTHMAN: Make sure we have got the 
right exhibit because I may be wrong. It's in Exhibit 
rs. 

Tavle??.4*oressxnibit'ls. You Nave, to do 
some adding up here. But, if you look, for example, at 
the primary sector, its share of GDP fell between 1970 
andi 20 “GromnG. 7 toss per "cent. It fell roughly in 
half And@tits! forecast’ ——’ so*that'"s "20 years. Its 
forecast to fall in the 20 succeeding years from 3.5. 
to 2.8 per cent, a Significantly slower rate of 


decline. 
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[10:50 a.m.] The manufacturing sector, as I suggested, 
has been roughly stable over that period of time. Its 
share fell from 1970 to 1990 from 26.9 to 25.5 per 
cent, that's partly cyclical and is forecast to stay 
roughly at that level, 25.5 to 25.9 per cent between 
then and 2010/2015. Those are the two most important 
parts of the goods producing sector. 

The construction sector is roughly 
constant through this peak, 6.7 to 6.4 per cent from 
1970; ton 90%.; andethen ,alfumther (falls fromté a4 storjust 
under 6 per cent in 2010, but that's roughly the same 
pace. So, it is that the rate of change is about half 
in the forecast period of what it had been in the past. 

If you take the other side of the coin, 
look at the services sector, it went from 50.1 to 56.5 
between 1970 and 1990, a gain of 6 percentage points 
over 20 years. From 1990 to 2010, the succeeding 20 
years, it increases by 2 percentage points. So, that 
is roughly half the rate of change. 

MReg SHEPHERD: 9 Q../fSosis atpfair to 
conclude then, Mr. Rothman, that the visual conclusion 
that you would make from a graph like this is, in fact, 
misleading? 

MR. ROTHMAN: A. As I said, it depends 


on where you put your line, your visual line, ona 
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gQraph like that. And that's, I think, why - if you 
don't object to my asking Mr. Mondrow to put the 
previous chart up-- 

Q. Okay. 

A. --I think the previous chart with its 
less compressed scale, is less exaggeration of this 
cyclical variation does give you a better idea of what 
in fact has been happening in the previous period and 
what the forecast is for the future. 

Q. Is this just accidental that this 
narrowly scaled graph was chosen or did you actually 
choose this because it is more representative of what 
you are forecasting? 

A. Well, as I said, I don't remember 
whether the person who plotted this chose this scale or 
whether I explicitly told that person what scale to 
use, but I can be quite explicit. 

If they had produced plot like the one 
you did, I would have told them, give me a plot with a 
less expanded scale because it doesn't convey as 
Clearly the visual picture that I want. This creates 
and impression of a great deal of instability in the 
series which I think has been much more stable, and as 
you suggest, can be looked at as implying that the 


forecast is only a continuation of the trend. 
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Q. It's a common problem in presentation 
of data in graphs, isn't it, that your scaling can 
affect the visual conclusion that you come to? 

A. Sure.) (And that’s why; as. Ftsaiay Mr. 
McConnell had a very strong preference for charts that 
started at zero. 

Q. I am not going to do both of the 
other examples, but I do want to do the next one, page 
SMOLMEXNIDVENEZGY 

Dr. Buja-Bijunas, this is your overheard 
NOSIS Ueh ExAibrCALo07 isn te se? 

DR. BUJA-BIJUNAS: A. That's correct, 
yes. 

Q. And I am going to show you that then 
rescaled ‘as/‘page 4 of Exhibit 126. ‘Now, in your 
testimony you described residential electricity 
intensity as flattening out into the future. Did you 
expect a significant change in the rate of residential 
intensity? 

ARP OPLUStT OL all, Iawould like. to 
emphasize that I didn't go into doing this forecast 
expecting a certain result in terms of intensity. I 
had expectations that there would be an impact on 
specific appliances due to standards. I had 


expectations regarding fuel share based on historical 
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fuel share estimates. I had expectations regarding 
particular parameters and variables that went into the 
forecast. How that all added up in terms of 
electricities per household I did not have specific 
expectations because that's not how you do end-use 
forecasting. 

Q. When you finish an end-use forecast, 
you do test it to see if the results make sense, don't 
you? 

A. When we finished, when we finally 
came up with the residential end-use forecast, we 
finalized it before we put together this graph. This 
gQraph was put together after the fact. 

O. .Thisis.outputwtrom youn model, not 
input. 

A. That's exactly right. We finalized 
the forecast first. When we finalize it we look at 
individual trends, individual parameters, 
interconsistency of parameters, rather than what 
expectations would I have regarding intensity of 
residential sectors. 

QO. If the output in terms of your 
intensity numbers in your end-use number were something 
that was totally unexpected, totally unusual, would 


that cause you to question the models or your inputs? 
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Let's say it had intensity dropping like a stone over 
the next 25 years. 

A. It would cause me to look back at 
parameters. If it was dropping like a stone, as you 
put it, I wouldn't go back and think, how do I increase 
it. What I would do is go back and recheck all my 
assumptions to make sure they are interconsistent and I 
am comfortable with them, and if, therefore, that is 
the result based on those parameters, that is the 
result. 

Q. It might be evidence of a problem, 
mightn't it? If you had an unusual output, it might be 
evidence of a problem with the model or the inputs? I 
am not suggesting it is in this case, I am asking as a 
general principle. 

fe SE Guess as’ 1 said sinemy Oridinal 
testimony at the very beginning, that when you look at 
the final kilowatthour per household, if it does 
something extremely unusual, you look back historically 
on what that ratio was as a comfort check. And so if 
it did do something extremely unusual, versus history, 
which I cannot explain, then I would not be 
comfortable. And whether that was due to the model or 
the data or whatever, I would to investigate what was 


the underlying reason behind it. 
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Q. Doctor, can you confirm that over the 
last 25 years on historical data, residential 
electricity intensity increased 70 per cent and you are 
projecting that over the next 25 years it will increase 
10 per cent? Can you confirm that that's about right? 

A. I would have to look at the exact 
figures but it sound about right. 

Ovezisett Lraireeetsave thatet hates” a 
fairly significant break from the past? 

A. In terms of growth rate, yes. 

Q. Am TI right in understanding that that 
result comes from all of the myriad of individual 
judgments you have made in the course of doing your 
residential end-use forecasts? You haven't sat there 
and said, well, I think 10 per cent would be a good 
number. 

A. No. The break is the result of two 
things. One is, what has gone into the future 
forecast, the other is, what gave rise to that growth 
historically. And a lot of that growth is due to where 
you are coming from. What I mean by that is, to go 
from a situation where nobody has electrical space 
heating to say 10 per cent of the population has 
electrical space heating, that's quite a dramatic 


increase. More so than if you go from 30 to 40. 
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I mean, the relative, what it would look 
like as an intensity looks as though it has more an 
effect on intensity if you are starting at a low base. 
And that's part of the accounting for why the 
accounting is increasing so much historically. You are 
starting off on a very low electrification level. 

Q. Now, do you do any analysis to try to 
disaggregate this sort of result, try to determine what 
the causes are for such a significant break, 
empirically, rather than, as opposed to, sort of 
getting a sense? Do you actually analyze your results 
and say the reason for this change is 50 per cent this 
result and 20 per cent this result? 

A. I don't have the figures with me but 
yes, when we looked at that relative difference, part 
of it is due to the relative difference in construction 
rates because the marginal fuel share for new 
construction is much higher than for the average stock, 
SO aS you have different construction rates that will 
impact your intensity. 

Part of it is due to the building code 
impacts, part of it is due to the appliance uptake. 
Both in terms of efficiency level and the actual 
penetration of appliances, which is due to differences 


in income levels previously as opposed to into the 
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forecast. 

Part of it is due to the difference share 
of types of housing. You have different energy 
requirements for single attached homes like row housing 
or duplexes or tri-plexes as opposed to single detached 
homes, and that relative ratio is changing. There are 
more row houses, semis and attached type of 
Single-family dwellings forecasted in the future due to 
demographics than historically, and that has an impact 
on electricity use. We have looked at the various 
things that would cause this change. 

The other major consideration too is this 
growth of "other" and everyone has been sort of 
focusing on this strong growth of "other." "Other" has 
actually been forecast to growth as a much lower rate 
than it has historically because within this "other" 
category they are end-uses such as TV sets, clothes 
dryers, supplementary secondary heating, additional 
baseboards, and lighting, and a lot of those are near 
Saturation level and therefore will not contribute to 
the intensity as strongly as they have in the past. 

So, all those things were considered in 
coming up with the forecast and all those things would 
lead to a kilowatthour per household level that would 


be less than it was historically. 
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Q. So, you are obviously very 
comfortable with this significant break from the past 
trend, aren't you? You know why it's happening. 

AS Ofiereiishawconsistent! scerarro that 
would explain why this would happen. 

Q.* Is anyot that’ changed,® that’ break 
from the past, related to -- let me rephrase this. 

How much of that change is related to 
expectations of tighter environmental or efficiency 
regulation? 

A. I will answer that in two parts, one 
which is the appliance standards, the impact of 
appliance standards, and we have already discussed the 
extent to whieh. 

Q. Yes. 

A. And we have incorporated identified 
Standards, standards that were identified at the time 
of forecast finalization, so that's going to have an 
impact because they would lead to greater efficiency 
improvements that would normally be the case. 

For end-uses where there are no standards 
identified, when it came to our efficiency assumptions, 
we looked at historical efficiency improvement in these 
particular end-uses and we did literature searches, 


looked at studies done by Lawrence Berkley Lab, et 
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cetera, to see what their judgments were regarding 
future efficiency improvements by end-use based on the 
sort of technologies that seemed feasible, what the 
costs would be of those technologies, what sort of 
Savings you would get from them. So, that is sort of 
the approach that we used. 

Q. My question was, you have historical 
time series data which shows a 70 per cent cumulative 
increase over 25 years and a forecast which shows a 10 
per cent increase over the next 25 years, cumulative, 
how much of that shift can you quantify, how much of 
that shift relates to increased environmental 
regulation that you have included in your forecast? Is 
it “Nalbjot the, shift! arthiird fofictthe shi &t? 

A. I guess I would have to calculate 
that > .4f wwould nave fotos 

o> 7 Cantyvourndorthabefor us? 

Aene ves... ieThetonly way) I>can’ think o£ 
really doing that is doing that is -- you are saying, 
what is the impact of the standards on kilowatthours 
per household? 

Q. Of increased environmental regulation 
and increased control of energy efficiencies. 

Ay” Yes; I -can do that. 


Q. Thank you. 
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I am going to turn now to the economic 
forecasts, and I don't really have a lot to ask about 
them, some specific questions. 

First, Mr. Rothman, I only have one small 
matter on the GDP forecast. You said quite a number of 
times that substantial increases in environmental 
regulation would have a long-term negative impact on 
GDP; that is to say, resources are directed away from 
the production of goods and services towards 
improvement of the environment which may be laudable 
but doesn't create domestic product. Is that a correct 
characterization of what you said? 

MR. ROTHMAN: A. I said it could, and I 
Said that that would be the likely direction. 

Os piDidn t woutan fact say that one of 
the reasons you have projected a decrease in 
productivity growth is the impact of expected increases 
in environmental regulation? Transcript reference is 
page 744, if you want to check. 

A. I think I know what that context was, 
but let me check it. 744 did you say? 

Q. 744 page, line 9. 


Ay TaYes* 
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[11:2,0.5,:4;.me)) <Q. sAmodm rightyein understanding that you 
have said one of the reasons why your productivity 
growth is lower than past history is increased 
environmental regulation? 

A. In that context it was talking about 
the results of the Ontec model, which I think were less 
explicit. But, yes, we have made some judgmental 
adjustments to productivity forecasts based at least 
partly on the expectation of the impact of 
environmental regulations. 

Q. Now, do you have any empirical 
studies or other hard data to show that increased 
environmental regulation will, in fact, reduce GDP or 
productivity? 

A. Could you repeat this specific 
question, please? 

Q. Does Ontario Hydro have any empirical 
Studies or other hard data that shows that increased 
environmental regulation will result in decreases in 
GDP or productivity growth? 

A. I believe there's an interrogatory 
response that carries some such results, and there has 
been extensive empirical work on the reasons for 
productivity growth slowdown it the '70s, some of which 


referred to the impact of the environmental regulation. 
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Q. So your answer to the question is, 
yes, you do have empirical data? 

AwiYes. 

Q. So this is not merely your intuition 
Or judgment; this is something that is proven by the 
time series or the historical data? 

A. Well, you can find frequently enough 
in the literature, studies that might go both ways, 
but, yes, there is empirical evidence that Suggests 
that increased environmental regulations can be 
associated with, can cause lower rates of measured 
output productivity. 

Q. Mr. Rothman, isn't it, in fact true, 
that this subject is the matter of some considerable 
controversy among the community of economists? 

A. AS I said, you can find studies that 
196+ 

THE CHAIRMAN: Well, no. Just a moment. 
Is it a matter of controversy? 

MR. ROTHMAN: Yes, I believe there is 
discussion. 

THE CHAIRMAN: Okay. You can go ahead 
now and elaborate. I just wanted to set that. That 
was the answer to Mr. Shepherd's question, whether it 


was a matter of controversy or not. 
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MR. ROTHMAN: Which is just to say that 
there are studies that say both -- go both ways. 

MR .1GHEPHERDssQs50TSsn!’t it,,ain fact, true 
that in the area of energy efficiency, certainly the 
most important aspect of environmental regulation from 
your point of view, the more efficient energy 
production and use, isn't it, in fact, true that, for 
the most part, you would expect a productivity 
improvement from that because you're using a resource 
more efficiently? 

MR. ROTHMAN: A. It depends obviously on 
the cost of the regulation. If you produce some 
regulation that requires a capital investment and one 
result of the capital investment is increased energy 
efficiency and therefore lower energy use, then there 
is some return to the capital investment danreccly. 

So, it is possible that will produce a 
lower cost than had you not made that investment. But 
it's a reasonable inference that if there were such 
investments available, that businesses would be making 
them without regulation. 

The reason for the regulation is either 
to accelerate or to produce such investments all 
together. And so it's possible that you can produce -- 


that you can lower costs by making these investments, 
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but it's more likely, in general, that if the 
constraint of the regulation is a binding constraint, 
that is, if it produces behavior that wouldn't have 
occurred otherwise, then that would increase costs 
because were it likely to decrease costs, businesses 
would have undertaken it without the regulation. 

Q. Do I understand you to be saying that 
in the business world businesses make decisions based 
on what is the economically best choice - in an 
economist's sense, the best choice? 

A. We could have long discussions about 
theories of the firm. 

Q. Do you believe that -- 

A. Let me satisfy the Chairman and give 
YOU: a VYEsqoOr=“nonanswer.— iam saying” that. "=P “am not 
Saying that businesses do that all the time about every 
decision, but I am saying that I believe that a 
reasonable description of the firm would be that over 
some period of time it makes rational decisions. 

I have long been a believer in the 
behavioural theory of the firm which suggests that 
firms, like people, have limited attention spans and 
limited ability to rationalize at any given time. But 
I think we've seen over the last ten years that firms 


whose attention span is so short that they don't get to 
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major decisions that are open to them, which would have 
a significant cost reduction, those managements don't 
stay in power. 

Q. What that suggests, Mr. Rothman, and 
correct me if I'm wrong, is that you probably don't 
need much in the way of efficiency regulation because 
if some change is truly efficient, then the business 
community will do it anyway; you don't have to tell 
them to. 

AyeGgNosnd Iasdidvat couple of lthings.. 1 
Said first, if the constraint is binding, it might 
simply accelerate investments that would have been made 
later anyway. That is, if an investment is efficient 
and does produce a positive payback, that's what the 
decision that the business is likely to have made 
anyway. But it might not have made that decision until 
the equipment broke down and a replacement was mandated 
by the life cycle of the equipment. 

A regulation could accelerate that 
decision and would therefore in the long run that 
regulation then would be neutral in effect on GDP. It 
would -- it might or might not produce a positive 
effect on GDP during the period in which it accelerated 
the investment. What we don't know is what the firm 


would otherwise have done with that capital. 
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And so I think that there is some effect 
of regulation of that kind. It accelerates investments 
that would otherwise have been made. Those kinds of 
regulations we might well choose to make in order to 
accomplish that acceleration. Other kinds of 
regulation are binding. They produce decisions that 
businesses or consumers would otherwise not have made. 

An example of an efficiency regulation 
that is binding is in the automobile industry. There 
has been a regulation that has bound the automobile 
manufacturers to reduce their gasoline consumption of 
their fleet, to increase their average fleet 
efficiency. What we have seen over the past five years 
Or so is gasoline prices have come down, is that 
despite the fact that the cars are, in general, more 
efficient, the average fleet efficiency of the cars 
being purchased has gone up. That's because as prices 
have come down, consumers have decided that they are 
less concerned with energy efficiency and more 
concerned with things like power or styling; or: size of 
the vehicle or whatever that is. 

That is one the kinds of thinks that I 
meant when I said that I don't see consumers as 
changing patterns but still responding to the standard 


variables of price and income, the standard 
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determinants of prices and income that determine 
consumer behavior. 

Now, I would say, therefore, that the 
average fleet efficiency regulations have been binding. 
They have caused an increase in efficiency of the total 
automobile fleet, but it's not what consumers would 
have chosen to do. 

Q. That's very interesting. Let me come 
back to electricity for a second. Do you believe that 
electric space heating is, in a commercial context, the 
economical decision to make if you're choosing a choice 
of one type of space heating? Isn't your testimony, in 
fact, that the life cycle costs of electric space 
heating are higher than natural gas? 

Aserres. -"hifercycle’costs 6ffelect rie 
Space heating are higher than those of natural gas. 
Those that the decision may be made to choose electric 
trying space heating for various reasons, including the 
fact that the person who pays the operating costs may 
not be the person who pays the capital costs. 

05> Seenotwan economic decision; 1s it? 

THE CHAIRMAN: I'm sorry. I didn't hear 
cnat’s 

MR. SHEPHERD. Q. It's not an economic 


decision, is it? It doesn't meet this paradigm that 
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you're talking about, about people making choices based 
on what is the most economic choice? 

MR. ROTHMAN: A. It does indeed. The 
decision-maker makes the choice about what is most 
economical for them. 

MR. BURKE: A. As we have indicated at 
various points, there are other factors than simply the 
operating cost or even the capital cost of the heating 
system that come to bear on the choice of the heating 
system, and, for instance, in the commercial sector 
there may be some value to the controlability of 
electric heating and its ability to be metered locally 
rather than having to be metered for the whole 
building ;sthat.ssort ,of thing. 

O.. vp Well peMr a Burke, gisn tt that true «of 
every area in which electrical efficiency is an issue, 
that there are other issues besides simply the 
efficiency? 

A. There probably are and some to a 
greater or lesser degree. I think you are proposing 
that there is purely economic decision? 

Q. No. I'm disagreeing with what Mr. 
Rothman is saying. 

A. Well, I think that the considerations 


in the economics can be broader simply than the capital 
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and operating costs. 

That is my point, that there is a value 
to consumers of some of these other aspects of fuel 
choice that is quantitative and, therefore, figures 
into their choice, and, therefore, the choice is more 
than simply a capital and operating decision. But 
there must be some value, some dollars and sense value, 
that consumers assign to these other benefits of 
electricity or they wouldn't be choosing it over gas. 

Q. So, Mr. Rothman, let's get back to 
the point then. Have I understood you correctly then 
to say that for the most part efficiency regulation 
will require people to either accelerate actions that 
would otherwise be good choices economically, or if it 
isn't just an acceleration, then it will require them 
to do things that reduce their overall productivity, in 
general? 

MR. ROTHMAN: A. Yes. As I recall my 
Original testimony on this, and it wasn't in the 
particular section you pointed to, but as I recall my 
testimony in direct evidence, I suggested that the 
effect of environmental regulation on economic growth 
would not be unambiguous, that there might well be 
regulations that would produce efficiency gains of the 


kinds that you're talking about. 
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My original testimony was primarily 
talking about end-of-the-tailpipe kinds of cleanup, 
that those things that reduce emissions tend to be 
capital intensive, tend to produce cleaner air, but not 
necessarily any or much additional measured GDP 
relative to the size of the capital investment that 


goes into it. 
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flilsa25am. | Q. So, you weren't including things like 
recycling of waste products and that sort of -- that is 
not the type of thing you were talking about? 

A. What I was saying was that on balance 
I would expect that increased environmental regulation - 
would have a large enough component of 
end-of-the-tailpipe kinds of things to produce a 
reduction in measured GDP and measured productivity 
growth, not that that was unambiguous. 

That there could well be parts of an 
environmental program or an efficiency program that 
would produce increases in GDP because they would 
discover more efficient ways of doing things than we 
had had in the past or would implement them faster as 
we have been discussing or would accelerate the 
implementation of more efficient ways of doing things. 

So, I am not changing what I said but I 
am not disagreeing with your point that there may well 
be such efficiency improvements available and that 
environmental regulations may point them out and have 
them adopted faster. 

Q. Am I right in understanding that 
you're contrasting things like scrubbers which are 
end-of-the-tailpipe sort of things and produce no net 


economic benefit, in fact, cost economically. And more 
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Structural things like recycling which will tend to 
produce an economic benefit. Is that a fair 
description? 

A. They might or might not produce an 
economic benefit in either of those directions. It is: 
possible that things like scrubbers or, in the pulp and 
paper industry, for example, recapture of chemical, 
could produce enough new material or reusable material 
that it would more than pay for its own cost. So it is 
possible that those kinds of measures too, could 
produce gains in productivity, not losses. 

It is possible that energy efficiency 
improvements could have such high capital costs that 
they don't produce enough increases in efficiency, in 
effect, to be worth their capital cost. It's those 
kinds of improvements that are those on which the 
constraint binds; that is, which would not be done by 
businesses without the impetus of regulation, and those 
would produce productivity reductions. 

So, it's possible on both sides. In 
general, I would think that it is more likely to get 
efficiency gains, productivity gains, from efficiency 
improvements. It is more likely to get productivity 
losses from ends-of-the-tailpipe solutions. But they 


are not mutually exclusive and not unique in either 
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direction. 

MR. SHEPHERD: Mr. Chairman, I want to 
pursue this for a few minutes longer, but perhaps this 
is a convenient time for a break. 

THE CHAIRMAN: You say you don't want 
ras 

MR. SHEPHERD: I do wish to pursue this 
for a few minutes longer, but I am going to start 
something new within this subject and maybe it would be 
an appropriate time for a break. 

THE CHAIRMAN: All right. We will take a 
break. 

THE REGISTRAR: We will recess for 15 
minutes. 
wae Recesseate bhaidiacn: 

---On resuming at 11:46 a.m. 

THE REGISTRAR: This hearing is again in 
session. Please be seated. 

THE CHAIRMAN: Mr. Shepherd. 

MR. SHEPHERD: Thank you, Mr. Chairman. 

Q. Mr. Rothman, I would like you to turn 
to Exhibit 129. Exhibit 129 is an excerpt from a study 
done by a group called DPA Group on measures to reduce 
greenhouse gas emissions; correct? This is the last 


four pages of conclusions. Have you had a chance to 
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Look vatathat? 

MR. ROTHMAN: A. Yes. 

Q. Is it fair to say that they conclude 
that the measures needed to achieve the environmental 
target are also a good idea in an economic sense; that 
is, they will produce a net economic benefit as well? 

AssuYes«o@Theyteonclide theresare 
efficiency improvements available that will increase 
economic efficiency as well. 

Q. That is a pretty common conclusion of 
economists in looking at environmental regulation; 
Pent hoe 

Av calawouldnyt gs= 

Q. Of various types? 

A. No, I wouldn't say that -- 

Q. You don't think that that's common? 

Ries §3NO:: 

Q. You spoke last week I believe - I 
don't remember the transcript reference - about the 
possibility of a tripartite free trade agreement 
including Mexico. Are you familiar with some of the 
considerations that Mexico is looking at in that 
negotiation? Have you followed that at all? 

A. I haven't had a lot of time to follow 


tha tines 
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Q. You haven't looked at the impact of 
environmental regulation in Mexico or their views of 
that? 

A,eo Nor 

Q. Before we leave this area, isn't it 
true that there are quite a number of other benefits, 
economic benefits, that you get from increased 
environmental regulation, aside from the potential of 
productivity improvement in certain areas? Isn't it 
true, for example, that you increase the attractiveness 
of Ontario as a destination for more highly trained 
workers, for academics, for scientists, et cetera; 
isn't that true? 

A. Yes. If you reduce -- these are some 
of the kinds of things that we had talked about earlier 
suggesting that although the direct effects of 
emmissions reductions, for example, are not included in 
GDP, there may be some indirect effects that might be. 
That's possible. 

Q. And when you you have things like 
that, like more academics and scientists highly paid 
people or highly trained people coming to Ontario, that 
would generally have an upward pressure on GDP; won't 
Ve? | 

A. I am not sure why you focus on highly 
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trained people and scientists. We have said that 
increased population tends to increase output. 

Q. Didn't you also say that as training 
improves, as skill level improves, productivity 


improves? 


Q. So, if an improved quality of life 
brings more people like that to Ontario, isn't that 
going to increase GDP? 

Agee sbedondt ‘know. /Sladont particularly 
want to split hairs over this stuff. I am just 
concerned that you seem to be saying somehow that 
Ontario wants to attract nice middle-class 
intellectuals only, and that is not really what we are 
Saying. 

Q. I am not sure I exactly said that, 
Mr. Rothman, and I guess I will repeat my question. 

The question is a very simple one: If you bring in 
highly trained workers as a result of whatever, but for 
example improved quality of life, is it or is it not 
true that as a general rule it will improve 
productivity and therefore GDP? Yes or no? 

A. Yes. 

Q. And isn't it true that a jurisdiction 


that has tighter regulation ina particular area, 
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whether it's environmental or anything else, will tend 
to develop indigenous industries in that area that will 
be competitive in other jurisdictions in that 

particular area, for example, environmental efficiency? 

A. In that area, it might drive away 
other industries. 

Q. That wasn't the question though. 

AgnaeYesasr 

Q. The question is if we have very tight 
regulations:-in area X, is it not true that as a result 
we are likely to have more efficient and more 
competitive industry producing the things you need for 
anea iX? 

A. Yes. And that might be a competitive 
advantage or sit might: mot.) Ifcother jurisdictions 
choose to have tight regulation in area X, then that 
could produce opportunities for further activity. Or 
it might simply be that those industries affected by 
tight regulation in area X leave and so you don't have 
any expertise at all in area X anymore, nor in its 
associated industry. 

It depends on how much tighter your 
regulations are relative to other jurisdictions and to 
what extent those other jurisdictions are interested in 


regulating in that area. 
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Q. If the practical circumstance is that 
Ontario becomes more efficient in a given area, has 
tighter regulation in a given area, before other 
jurisdictions but they ultimately catch upy tence it 
also true that we have an edge when they catch up? 

A. As I said, it depends on what the 
responses are of the industries involved. 

Q. Ontario enacts a regulation that says 
that all new industrial motors will be variable speed 
high efficiency motors. Ten years later, New York then 
enacts a similar regulation. Is it true that ten years 
later, we have an edge, a competitive edge? 

A. That's not necessarily true, no. Are 
you suggesting that we have a competitive edge in the 
Production of energy-efficient motors? 

QO. | |Wellyeiboth: Bue) might new fam 1 
talking about the industries that have already shifted 
onto the new regulation have an edge when somebody else 
is trying to shift; isn't that right? 

A. It might or might not. Depends on 
whether the requirement that they buy energy-efficient 
motors has prevented them from making other capital 
investment that would have increased their efficiency 
even more than would have bought buying 


energy-efficient motors. 
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Some industries may leave Ontario because 
it's too expensive for them to buy the energy-efficient 
motors. Some industries may comply with the regulation 
and buy energy-efficient motors and then be more 
efficient than their competitors elsewhere. 

Some industries may buy energy-efficient 
motors and stay in Ontario but become overall less 
efficient than their competitors elsewhere because 
their competitors elsewhere are buying other kinds of 
efficient productive equipment that gives them an edge 
over Ontario industries who are only re-equipping their 
motors, and who have been forced to do so by 
regulation. 

Meche producttror of smotors, rE Ontario 
already has an industry producing motors, then we might 
well have some competitive advantage in the production 
of energy-efficient motors assuming that that industry 
converts. But it might simply be that we buy the 
energy-efficient motors from outside Ontario anyway, 
and that doesn't give us any particular edge. 

Q. I get the impression, and please 
correct me if I am wrong, I get the impression that 
what you are saying is we can have more tighter 
efficiency regulation and environmental regulation and 


it may indeed be something that we would want to have 
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because it will make Ontario a nicer place to live, but 
we are going to have to pay for that ultimately in less 
economic growth. You don't get anything for nothing. 

A. I am saying that is what I believe to 
be the most likely outcome of such regulation. I am 
not saying that there couldn't be regulation of a kind 
that would produce higher economic growth and increased 
energy efficiency at the same time. 

But it is my judgment and my expectation 
that the kind of regulation that we will get will on 
balance reduce total of economic growth. And we will 
get a better environment for it, but it will reduce 
measured economic growth in order to do that. 

Q. And as you have said, this judgment 
that you have is quite a controversial judgment within 
the economic community; is it not? 

Ay) Lydon st. vbhink) sikGetish ta) ethat 
controversial that end-of-the-tailpipe kinds of 
regulation reduce productivity growth. That, I think, 
is reasonably well-accepted. 

Q. You believe there is a consensus on 
that? 

A. The economic community. 

On the question of energy efficiency 


improvements and whether many of those available which 
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have positive paybacks in and of themselves without 
regulation, I think there is some controversy, so many 
of them available that we could get a significant 
increase in efficiency and therefore output, I think 
there is some discussion within the community, yes. 

Q. Now you have included in your GDP 
forecast some increased environmental regulation, 
haven't you, as an assumption? 

A. Judgmentally, yes. 

Q. Can you quantify the amount that that 
has reduced your projection over what it would have 


been? 
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[12:00 p.m.] A. It's hard to be precise. I believe 
that I testified that the LISA model would have 
produced a rate of productivity growth a little under 
half a per cent per year higher than what is in the 
forecast. One of the reasons for reducing that model 
forecast, judgmentally, was the expectation of an 
increased environmental regulation. 

Q. That's not the whole half of one per 
cent? 

A. That's not the whole half of one per 
cent: 

QO. 'olseit aglittleGbit lof lallottebert, 
half of it? Can you give us an idea? Are we talking 
about a significant component of your forecast? 

A. It's hard to quantify. I would say 
when you say a minimum of .1 per cent of the half per 
cent and maybe... 

THE CHAIRMAN: .1 of the half per cent? 

MR. ROTHMAN: Yes. So maybe .1 per cent 
per year in growth is the minimum. 

THE CHAIRMAN: That would be .2; wouldn't 
Ls 

MR. ROTHMAN: Pardon me? 

THE CHAIRMAN: It's a fifth of it anyway. 


MR. ROTHMAN: A fifth of it, yes, ora 
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quarter of it, because it's really .4, I think. 
To what, .2 or.3, something in that range. It's hard 
to quantify that judgment. 

MR. SHEPHERD: Q. So, you didn't examine 
it expressly, you had a number of factors that made you 
think the LISA projection was high. And so you pushed 
it down because of the balance of all of the factors? 

MR. ROTHMAN: A. Yes. 

Q. So, you couldn't go back then to your 
work and give us an actual number for what the impact 
of increased environmental regulation is on your 
forecast, you couldn't do that? 

A. No, but we are working on it. 

Q. On a new model that will do that or a 
new technique? 

A. We were working on some modifications 
to the LISA model that will allow us to better quantify 
the impacts of environmental regulation on GDP growth. 

Q. So, there is a potential here that if 
the economists who disagree with you on this point are 
right. There is a potential here that your GDP 
forecast is quite a bit too low, isn't there? 

A. What do you mean by "quite a bit"? 

Q. Well, it seems to me that if you 


compound .2 of 1 per cent over 25 years you get, in 
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absolute terms, a relatively big change in GDP at the 
end of the 25-year period; isn't that right? 

A. Well, .2 per cent comes to, what, 5 
per cent or so. You get a noticeable difference in 
GDP, yes. We were talking about that earlier. 

On the other hand, there is a real 
potential that our GDP forecast is too high, that in 
fact, here would be significant regulation of the 
end-of-the-tailpipe variety and other regulations 
forcing more expensive technologies. 

Q. Now, these things, a question like 
this, that is captured in your uncertainty bandwidth on 
GDP, isn't it, or on the implicit uncertainty bandwidth 
on GDP in your model? 

A. Yes, I think. If we had a very 
Significant break in trend, I am not sure that that's 
Captured by the uncertainty model. 

Q. And that's a break in the trend that 
you have assessed to be there, which is, as you said, 
is not one that everybody necessarily agrees with? 

A. I am not sure what you mean. 

Q. Don't I understand correctly that the 
trend you have projected would have environmental 
regulation reducing GDP, not everybody agrees with 


that, correct? 
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Asstrey es 

Q. So, a break in that trend of your 
judgment, that is if the other people are right, or if 
you're right but you are projecting the amount of 
environmental regulation wrong, either of those things 
would take you outside of your bandwidth; wouldn't it? 

A. It depends on the degree to which we 
are talking. If we are talking about a .1 per cent 
difference or .2 per cent difference, that would be 
covered by the bandwidth. But if we are talking about 
essentially moving, implementing regulations that wind 
up taking us towards zero GDP growth, then that would 
be a break in the trend, it would not be covered by the 
existing uncertainty bandwidth. 

Q. Now, implicit in your bandwidth 
methodology, if I understand correctly, is the notion 
that there is a less than 10 per cent chance that 
either of those two those things will be true; isn't 
it? Either the significant break that you haven't 
projected from an assumption point of view, or that the 
other forecasters would be right in terms of the impact 
and you would be wrong. Combined, you are saying there 
is a less than 10 per cent chance of that; isn't that 
raght? 


A. A 10 per cent chance on each side. 
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Q. Except that they both move in the 
same direction. So if you only moved in one direction, 
there is only a 10 per cent chance on that side, isn't 
that right? 

A. Perhaps I wasn't hearing you quite 
right. I thought I heard you say that people who are 
Saying that there is increased efficiency improvement 
available, that would be on one side and those who -- 
what also went on that side? 

Qoe il mesorryPiyou "re sright. S°v6a?are 
cormect. 

It's the one impact, whether you're right 
orthe other economists are right, would move you in the 
direction of higher GDP; right? 

A. Yes. 

Q. You are saying that there is a less 
than 10 per cent chance that that could be true? 

Aw Wess 

Q. And the other impact, a significant 
break in the amount of environmental regulation would 
on your assumptions take you to a lower GDP and you are 
Saying there is a less than 10 per cent chance of that? 

As Os¥es« 

Q. Let me move on, then, to an area that 


has a number of impacts and that is interest rates. 
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First, this is your area, Mr. Rothman, 
correct? 

Pies 1 LES 

Onyi7 Could*youygivesusbas list2oreaqust tell 
us the various major areas in which your interest rate 
projections affect the DSP? 

A. As as far as I understand, the most 
important area in which the interest rate projection 
affects the DSP is in the calculation of the discount 
rate that's used for calculations of present worth. 

Q. Well, the interest rates are 
obviously included in your economic forecasting, right, 
your econometric models? 

A. I would have to check the models, but 
I believe that our LISA model is a real model. I don't 
think it has a monetary side directly. And if it does 
have interest rates, I think interest rates are 
endogenous; that is, I think interest rates are 
determined within the model rather than being fed into 
it aS an assumption. 

Q. So, that would mean it wouldn't 
necessarily be -- the interest rates that come out of 
the model, if you like, that are used to drive your 
GDPaaas SOrlty. 


Bs. «SOELY; 
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Q. The interest rates that are part of 
the calculation of your GDP results, then, are not 
necessarily the interest rates that you have projected 
separately and used by Ontario Hydro, is that right? 

If they are endogenous to the model, then 
they are not the same ones that you have projected, are 
they? 

A. If they are endogenous to the model 
and they were projected, then that's what we would have 
in our forecast. 

Mr. Burke reminds me that there is a 
investment function in the LISA model and it does have 
interest rates on the right-hand side. So I am going 
to have to back off that and say that I would have to 
look again at that model to know what role interest 
rates play infiie. 

MR. BURKE: A. Maybe I could just add, I 
think that in the long run the GDP value itself is not 
determined by interest rates; it's determined through a 
production function-type approach to output in the 
Ontario economy. 

Interest rates would perhaps affect the 
Shares of output in the long run. 

Q. Well, Mr. Rothman, haven't you 


testified that one of the factors in GDP growth is 
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capital formation? 

MR. ROTHMAN: A. Yes. 

Q. And isn't that affected by interest 
rates? 

Agpriachinkliters intithe model, that's 
right. 

Q. I am just surprised that you don't 
know whether a factor such as interest rates is even in 
your model or not. 

A. I don't run the model myself. 

Q. This wasn't actually a question that 
I was intending to make a point on, I thought it was a 
given. 

if its tdaefiiienititonputivourvtingerson, 
there is really no point to be made from it anyway, 
maybe we could just go on. 

A. Let me just check it. There is a 
long-term corporate bond rate as an explanatory 
variable in the business investment and machinery and 
equipment equation, and that corporate bond rate, 
according to my variables list, is a McLeod Young Weir 
bond rate obtained from DRI. 

O; iSO, eit\seanot eeiedvin any, way to your 
forecast of interest rates? 


A. The historical data used to estimate 


Farr & Associates Reporting, Inc. 


Rothman, Burke,Buja-Bijunas 2078 
cr ex (Shepherd) 


it is from DRI. It's f6recast osbe cur forecast, a2 
presume. 

Lawild sjustecheck “thates ltrs tan 
exogenous variable. 

Q. You also use interest rates in your 
end-use modelling? 

DR. BUJA-BIJUNAS: A. Interest rates are 
used in the INDEPTH model for the industrial sector. 
Again, for the investment function, one of the things 
that tet 1is6ks at is for a given capital expenditure, 
typically, what percentage of the monies required have 
to be borrowed and therefore what interest rates apply? 

But the interest rate used there is done 
in a sort of simplistic way. You can only ascribe one 
interest rate for the entire forecast period. The 
input variable cannot be changed for each year of the 
forecast, so we just sort of use an average interest 
rate over the forecast period. 

Q. COMMEND and REEPS don't use interest 
rates t+atvall? 

Rh. SNe ethey*den*t?” Itt vs Fust “the 
industrial INDEPTH model that uses an interest rate. 

Q. So then, in all of your forecasting 
activities, in no case do you actually use your own 


interest rate projections, do you? 
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A. Oh, no. The interest rate is our 
interest rate. What I am saying is that the model 
doesn't allow you to put in every year's interest rate; 
all I can do is put an average interest rate over the 
entire forecast period. 

O.n= Youccant  puts#Hydro'sinterest rate 
forecast into you are model, can you, because the model 
won't take the whole forecast? 

A. It will not have a yearly interest 
rate; it will just an average over the forecast period. 

Q. And, in fact, in two of your models 
you don't use interest rates at all? 

Aeeaethat’s correcct:. 

Q. Is there an assumption there that 
those intersection end uses are not sensitive to 
interest rates or is it just a limitation on the model? 

MR. BURKE: A. I just might add. It's 
my understanding of REEPS in terms of choices, there is 
an implicit discount rate for end uses where the 
discount rate is relevant. In the trade-off between 
Capital and operating costs, the choice equations 
embedded in REEPS implicitly estimate the discount 
rates used by consumers in making their choices which 
may not necessarily correspond to a Hydro interest rate 


of some kind. 
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It may be appropriate, for instance, in 
some energy market choices that people effectively look 
for two or three year payback and that comes out 
empirically in the coefficients that are applied to the 
Capital and operating cost terms in the choice 


equations. 
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[10:14 am] So, it's not an Hydro interest rate. 
It's an implicit discount rate in those choice 
equations where they exist in REEPS and COMMEND. 

Q. Now, Hydro does, in fact, use your 
projection of discount -- or of interest rates and 
therefore discount factors for assessing the capital 
costs of central generation projects, doesn't it? 

MR. ROTHMAN: A. Yes. 

Q. And the viability of demand-side 
management initiatives? 

A. Yes. 

Q. And the amount of economic 
non-utility generation? 

AveoYes. 


QOa9 Insfacty) fall’ ofeats avoided cost 


2081 


calculations in its comparison of one option to another 


all use your interest rate projection; right? 
A. Yes. 
Q. Your forecasts don't use your 


projections but your planning does? 


A. Our forecasts do use our projections. 


Q. Well, sorry, didn't you say that your 


your econometric model uses McLeod Young Weir's 
forecast; not yours? 


A. No. No. No. I said that was a 
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McLeod Young Weir series, historical series, that we 
get from DRI's data base, and we use that for our 
historical estimation of the structure of the model. 
And for the forecast, we attune the DRI model to our 
forecast and use that input. 

Q. Sorry. I misunderstood. So you do, 
in fact, use your own interest rate projections in 
LISA? 

Lee * "LES" 

Q. Okay. In the end-use modelling you 
don't, but in the the econometric modelling, you do. 

MR. BURKE: A. I think Dr. Buja-Bijunas 
has just said we do use the official corporate interest 
rates that we forecast in INDEPTH, but they cannot be 
entered on a year-by-year basis, so they are proxied by 
the average over the period. And probably from the 
perspective of the sorts of decisions that are being 
made within INDEPTH, there's no great loss in accuracy 
as. a. results of thats 

Q. Now, you project real interest rates 
and inflation rates separately, don't you? 

MR. ROTHMAN: A. Yes. 

Q. And then when you need nominal 
interest rates, you multiply the two together;:.is*that 


correct? 
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A. Effectively, yes. 

Q. Sometimes you add them, sometimes you 
multiply them, but the number is roughly the same 
anyway, isn't it? 

A. Yes. 

Q. So, if you project 4 per cent 
inflation and 3 per cent real interest rates, then 
you'll get 7-1/2 per cent nominal interest if you add 
and 7.6 per cent in you multiply; is that fair? 

A. Something like that. 

Osukt te saprettyt close, Sisni eo itt 

If your nominal interest rate 
projections, or, indeed, your real interest rate 
projections, I guess more to the point, were to change, 
what effect would you expect that to have on a large 
Capital-intensive central generation facility in terms 
of the capital cost of that facility? 

A. The higher the real interest rate, 
the higher the lifetime cost of any capital-using 
faci kit ye 

Q. Is there a rule -- 

A. The more capital-intensive it is, the 
higher its relative cost will become. So, what happens 
is, aS real interest rates increase, the relative costs 


of the more capital-intensive facilities increase. The 
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relative costs, that is relative to other alternatives, 
of the less-capital intensive alternatives decrease. 

Q. Is there a rule of thumb for relating 
the capital cost of a project to the increase in real 
interest rates? 

Pot eg don’ ty icnow ofitone. 

Q. I've been told -- correct me if I'm 
wrong -- that commonly you would say a l per cent 
increase in interest rates -- 

MR. B. CAMPBELL: Hasn't he answered this 
question? He says he doesn't know of one. I don't 
see how it can be -- 

THE CHAIRMAN: Well, he can ask this 
question. He's been told. And if he knows about Lat, 
he can answer it. 

You've been told what, Mr. Shepherd? 

MR. SHEPHERD: Q. Yes. You can tell me 
I'm wrong, and please feel free, that al per cent 
increase in interest rates would typically produce a 10 
per cent increase in the capital cost of a -- or the 
lifetime cost of a capital-intensive DEOaect. Is that 
sort of a rule of thumb that you've seen in the 
industry? 

MR. ROTHMAN: A. I haven't seen it used. 


It might or might not be a valid rule of thumb. 
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Oboe iCkay: 

A. Remember, we're not in the business 
of projecting capital costs. 

Q. Okay. So, you wouldn't know what the 
impacts are quantitatively of your interest rate 
projections, whether they're higher or lower? You 
wouldn't know the impacts from a capital cost point of 
view? 

Awis NoOvyimotudrrectly. "tis! a.difterent 
part of the organization and it -- in a sense that 
question gets back to what we were talking about 
earlier. They don't come to us and say, do you realize 
that you're increasing real interest rates? Just 
increase the, cost ofic/Yisactivity) bya"x°= bilTion 
dollars. 

Q. Do I understand your previous 
testimony correctly that in your view lenders got hurt 
in the 70's because of high inflation, and as a result, 
they required an increased real interest rate premium 
in the 80's? 

A. From unanticipated high inflation, 
yes. 

@: And so that's the result, that they 
asked for higher real interest rates in the 80's; is 


that correct? 
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A. Yes, and empirical evidence 
Suggestion that falling inflation produces higher real 
interest rates. 

Q. And you are projecting that this is a 
structure shift in the economy, aren't you? That is, 
that that expectation of higher inflation will 
continue? 

A. No, but that the lenders continue to 
get a premium for the possibility of higher future 
inflation especially as inflation continues to falt, 
and that therefore real interest rates stay high 
relative to those... 

Q. Mr. Rothman, I'm going to refer you 
to page 327 of the transcripts. It's in Volume 2. 
You've said at page -- sorry. At line 3: "Our 
forecast is for real interest rates to stayehigh “for 
some time but to start to fall in the mid-1990s." 

And later on you've said, in effect, you 
were the first ones to forecast those high real 
interest rates, and now everybody else is starting to 
agree with you. Is that sort of the sense of it? 

A. Yes. I didn't say "the first?" but 
among the first. 

Or Wes it¥es | i fOKaye tino ; your interest 


rate projection then, if I understand this analysis 
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correctly is dependent on low and stable inflation, 
PSASERIC? 

A. Lower and stable inflation, yes. 
Well -- yes. 

Q. If inflation is unstable, if it moves 
around like in the 70s and 80s, then your conclusion is 
that you'll have much higher real interest rates; isn't 
that true? 

A. We would get a different pattern. If 
inflation started to move up unexpectedly, real 
interest rates would fall initially but would then rise 
again. | 

Q. Your overall average would be higher, 
wouldn't it? 

A. I would expect that it would be. 

Q. Substantially? If you had a reapeat 
of the 70s and 80's -- let's just take the 70s and 80s 
and we'll move them and make them 90s and the first 
decade of the next millenium. Would it be fair to say 
that you would anticipate higher average real interest 
rates, substantially higher, than you have currently 
projected? 

A. In the first few years of that 
scenario I would expect lower real interest rates. [In 


the first five years, I would expect you would get 
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lower real interest rates. Over the next ten years, 
say, I would expect higher real interest rates, and 
over the 15-year period, I would expect real interest 
rates to be higher, GDP growth to be lower than in our 
Forecast. 

I would find it hard, off the top of my 
head, to put a number on that partly because of the 
necessity to put numbers on it to have some more clear 
idea of what was going on and to look at it more 
carefully. But directionally I think that's where we 
would go. 

Q. I'm showing you page -- why don't we 
just go to page 6 of Exhibit 126. Now, this is 
actually page 5 -- isn't it? Maybe I'm screwed ep. 

DR. CONNELL: Page 7. 

MR. SHEPHERD: Page 7? You're right. 

Q. Well, this is one chart of your real 
interest rates forecasts. Let's look at page 8 though. 
It's probably easier to follow. I'm not trying to be 
misleading by choosing 8 over 7, as you'll see from the 
point. 

Now, that spikiness that you see there in 
interest rates in the 80s, that is a result of the 
instability of inflation in the 70s; isn't that right? 


MR. ROTHMAN: A. Some of the spikiness, 
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the real spike that you see there, is a spike in real 
primes in 1990, in the late 1980s and into 1990. And 
that real spike is a function of a very tight monetary 
policy pursued during that period. 

Q. It wasn't really market factors so 
much as the government just kept the rates higher? 

A. Yes. 

Q. But if you look at == the important 
figure for Hydro is the long bonds figure, isn't it? 

A. 40Yes; 

Q. And if you look at the long bonds, 
you have in the 80s relatively higher rates and much 
more volatile rates. And is it true that that is 
essentially the result of inflation volatility in the 
70s and 80s? 

Re Wellpewhateyouchavelis amcyclical 
pattern there in in long bond rates with a peak in '81, 
which was just before the start of a recession, or at 
the start of a recession, not unusually, and a fall 
during the recession, then a peak during the recovery, 
and then a gradual decline from there -- a peak during 
the very hotest phase of the recovery. 

But I would agree that the height of the 
peak and some extent of its volatility was due to the 


volatility in the 80s, yes. 
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Q. Now -- 

AsOuSorgye! OftinElation in theé*70s: And 
the 80s, actually. The inflation peaked in -- I think 
the actual peak of measured inflation was in '81 or so 
and then -- 

Q. Why don't you turn back to number 7. 
I don't want to perceive to be misleading you with the 
chart. There's a trough there in the 70s, very low 
real interest rates. Right? 

A. Yes. 

Q. Am I right in understanding that's 
because investors didn't anticipate the extent of 


inflation and so got caught unawares, as it were? 
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L233 pe | Ae Nettherritstherght cnoretts 
persistence, that's right. 

Q. Do I understand your testimony just a 
minute ago to be that if you had the same sort of 
volatility you would have the same sort of trough in 
the '90s? 

A. I don't think you would get as far 
down in the '90s for things like the prime rate. But 
yes, I think if you were to have a significant 
acceleration of inflation in the early '90s that you 
could get a trough of real prime and real long-term 
interest rates again. | 

Q. You wouldn't anticipate that long 
bonds would go as low as they would in the '70s, 
though, would you? 

A. I would not anticipate that. 

---Off the record discussion. 

MR. ROTHMAN: No, I would not anticipate 
that. 

MR. SHEPHERD: Q. As I understand what 
you are saying, the investors in a stable period of 
inflation feel they have a pretty good grasp on what 
future inflation will be, so they can make their real 
interest rate margins finer, because they are 


comfortable that they are covering inflation first; 
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right? 

MR. ROTHMAN: A. Yes. 

Q. But if there is volatility, then they 
want to make sure they get the real interest rate so 
they sort of have a higher expectation of long-term 
inflation; is that right? 

Pose ALESi 

Q. And that means that nominal interest 
rates are higher? And real interest rates are higher? 

A. Yes. 

That's not the only factor. The 
inflation volatility and ay risk of higher inflation 
in the future is not the only reason for our forecast 
of relatively high real interest rates. 

Q. I was going to come to that. You are 
also anticipating some number of other factors that are 
going to keep interest rates Up; FYonE ? 

A. Yes. 

Q. Government deficits? 

A. High levels of government debt 
relative to GDP -- GMP. 

Q. Just as an aside, you did these 
forecasts before the latest provincial budget; didn't 
you? 


A. Yes. 
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Q. Did your forecast contemplate the 
level of deficit that was in the current budget? 

Ae No. 

Q. Is that a significant enough change 
to make a difference? 

Aaa Wer will Mlooky atudt. (oT don'ts think so 
at. thisepotnt:. 

Ose Doesmthat mean rt rs? not significant 
enoughya fiatriis® one?) time! or Wt"'s: not’'significant if it 
is a change over several years? 

A. I think Yi it? is one’ time it''s 
unlikely to be significant. If it's a change over 
several years, we will have to look at it. 

Q. You have no sense now as to whether 
it is significant? 

A. We are looking primarily at levels of 
federal government debt to GDP; and relative to that, 
the Ontario government deficit is not large now. 

Q. You are not saying you don't look at 
the Ontario Government situation, but it is not as 
important a factor? 

A. We haven't looked at it in the past 
because the Ontario Government has not had Significant 
amounts of debt on the market, so we have not looked at 


that question of whether their debt levels would affect 
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the aggregate level of debt, government debt to GDP. 
But we will now, I think, have to look at it, yes. 

Q. Most Ontario debt in fact is Ontario 
Hydro as opposed to Ontario government; isn't it? 

A. Yes. 

Q. If we don't have the sort of 
Stability of inflation that you are anticipating in the 
‘90s and the next two decades, those other factors, 
deficits, Kuwait, Eastern Europe, all those things you 
talked about earlier would tend to push real interest 
rates up; wouldn't they? 

If you don't have the depressing factor 
of stability of inflation, which you have said tends to 
keep real interest rates down, then those other factors 
will have freer rein to increase it; won't they? 

A. Whether or not we have stable 
inflation, those are factors tending to produce high 
real interest rates. 

Q. When you provide discount rates to -- 
I will leave .that..I'll get to»itvlater)« ‘bets go to 
natural gas prices. This shouldn't take Long ?yerit is 
the last thing before we start on uncertainty. 

I am showing you overhead No. 10, page 10 
of Exhibit 126. And for the record I believe this is 


No. 7 of Exhibit 100. This is your graph in fact; 
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PONE balks 

Piss oti NOS. 

MR. B.. CAMPBELL: Sorry, is this Mr. 
Rothman's? It would be Exhibit 99, I believe, if it 
was used by Mr. Rothman. 

MR. SHEPHERD: My question was sort of 
MUIEL—-directed, «S0.-:.+ 

DHE. GCRATRMAN: .dwmthainkoit is.Exhibit.100. 
It looks like it. 

MR. SHEPHERD: ..Oiaendthisvyiceayvyours,tMr. 
Burke? 

MR. BURKE: A. Yes, that's ee 

Q. Now you testified, I think, that 
natural gas prices have historically been about 50 to 
70 per cent of electricity prices when you measure them 
in terms of energy services delivered; isn't that 
right? That sort of range? 

A. I am not sure I put a number on the 
record, but that sounds about right. 

Q. Just the right range? 

A. Yes. For the residential sector. 

Q. Okay. Is there a difference in other 
sectors? 

A. Yes. Essentially in the residential 


sector, the prices are, if anything, closer together 
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than they are in the commercial and industrial sectors. 

Q. Now, the effect of that fact is that 
natural gas has a price advantage, right, a very 
substantial one? 

A. Looking at pure fuel costs, it has a 
price advantage, yes. These are efficiency adjusted by 
the way. 

Q. Yes I understand? 

A. So that this is not straight market 
prices we are looking at here. 

Os nes is measured in terms of energy 
services delivered; right? 

A. Yes. 

Q. That's what this is as well; isn't 
LE? 

A. Yes. 

Q. I am going to take you then to the 
next overhead. This is an extrapolation from the data 
underlying your overhead and your projections of 
Natural gas prices. And what it attempts to show, and 
I should tell you that it SayS it shows the percentage 
by which electricity price exceeds natural gas price, 
but, in fact, if you look at the y axis, that's a 
Propurtion simply because I haven't figured out how to 


tell Quattro Pro how to write it in percentages yet. 
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So you can read those as percentages. 

So this is intended to show the size of 
the price advantage that natural gas has. Is that 
clear and understandable? 

A. Yes, I think so. 

DR. CONNELL: Excuse me, it is not clear 
to me what the Y axis is. 

MR. SHEPHERD: The Y axis, actually you 
should read it as percentages; that is 20 per cent, 40 
per cent, 60 per cent, 80 per.cent which is the amount 
by which the electricity price exceeds the natural gas 
price for the same energy services. 

DR. CONNELL: Thank you. 

THE CHAIRMAN: This is derived from the 
previous graph? 

MR. SHEPHERD: It's derived from the 
previous graph and from Mr. Burke's testimony as to the 
changes in prices that he is anticipating in 
electricity and natural gas. 

MR. BURKE: I believe it was Mr. Rothman 
who spoke to the changes in future prices. 

MR. SHEPHERD: I am getting totally 
confused, but one of you... 

MR. BURKE: I used this overhead to talk 


about elasticities and their -- 
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MR. SHEPHERD: I understand. 

Q. So Mr. Rothman, then, is this a fair 
representation of the changing relationship that you 
see between electricity prices and natural gas prices; 
that is, that over the next 25 years, the price 
advantage of natural gas will fall quite substantially? 
is that fair? 

MR. B. CAMPBELL: I'm sorry. You are 
including the efficiency adjustment effect in that 
question, I assume? 

MR. SHEPHERD: Yes. 

MR. ROTHMAN: I think that's right, yes. 

MR. SHEPHERD: Q. So, if I understand 
correctly - and let's just accept for now that the data 
is correct and please feel free to come back and 
correct it if you feel the underlying data is wrong - 
that a current price advantage in the order of 130 per 
cent is going to drop over the next 25 Yeats ttélan price 
advantage in the range of 45 per cent? 

MR. ROTHMAN: A. Yes. That looks about 
right. 

Q. Now, Mr. Burke, let's come back to 
elasticity for a second. You testified that you found 
very little cross-price elasticity between natural gas 


and electricity» right? 
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MR 3. BURKE: ..Asp Lesaid,that..on. neti ,I.got 
very little cross-price elasticity. There was a 
positive effect in the residential sector anda 
negative effect in the industrial sector, and they 
offset each either so the total system demand was -- 
there was no cross-price effect in the long run. 

Q. Now haven't you also testified that 
in some areas, for example, own-price elasticity, your 
data set doesn't allow you to be confident with the 
elasticities beyond a certain. point. That,is),) if the 
change is too much, you are not confident with your 
elasticity numbers? You have testified to that in -- 

A. Yes. And we have also testified to 
the extent to which changes in cross-price 
relationships, certainly ones in which the gas price 
moved into very close relationship or started to exceed 
electricity that that too would lie outside historical 
experience and we would not be confident, that was 
include in my direct evidence. 

Q. Is this an example of that? Is this 
getting close enough-- 

A... No. 

Q. --that you would have doubt about 
your elasticities? 


A. No, because as you can see, we have 
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had, by the end of period we get back to the experience 
of the early '80s, and we are in a situation where 
natural gas has still got a 45 per cent, as you “put it, 
competitive advantage over electricity, and probably 
the ‘sorts’ of factors ‘that are affecting interfuel 
substitution between electricity and gas are, I think, 
Still within the range of historical experience then. 

Q. So then you would say that ina 
future that you are projecting like here, you would 
anticipate that despite the size of the relative price 
change that there would be basically no price-induced 
net fuel shifting; isn't that right? 

A. Yes. 

Q. Can you tell us the basis on which 
you concluded that natural gas prices and electricity 
prices would move so close together? 

MR. ROTHMAN: A. Well, we do forecast 
increases in real electricity prices now, through this 
period, but we have a forecast for a faster gas price 
increases than electricity price increases. 

That is on the basis of a forecast of 
increased gas use in North America, significant 
increases in gas use for electricity generation both in 
the United States and in Canada, most importantly in 


the United States, an increase in the integrated gas 
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market in North America, with new pipeline capacity 
going from Canada to the United States so that Canadian 
gas users have to compete with U.S. gas users for that 
gas. The consequence of that is a depletion of the 
existing resources and a need to turn to more expensive 
resources to supply this growing market. 

Q. Is this a forecast based -- it is not 
a forecast based on extrapolation from past data; is 
it? It is an analytical forecast? 

A. Well, yes. 

QO-i gtoumhave, nine Pack)" said that’ you 
don't use any formal models in this; haven't you? 

A. We use a mixture of models and 
judgment; but we don't have a single formal model, 
Enatws right: 

Q. And you have also testified that it 
is based largely on judgment; haven't you? 

A. Yes. There is a large element of 
judgment in that forecast. 

Q. Now is this a consensus of 
forecasters? Is that how you do it? 

A. No. It's a forecast essentially by 
our fossil fuel analyst. 

Q. You have one person doing it? 


A. One person. 
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Q. And it is that person's judgment that 
is relevant here? 

Ast) Yess 

Q. They are the only person making these 
judgments. 

A. Primarily. We are certainly in 
contact with other forecasters and we subscribe to at 
least two external forecast -- actually, three external 
energy forecast services. We use those forecasts as 
well and we discuss forecasts with other forecasters, 
but this is essentially that analyst's judgment 


informed by all of this information that he can get. 
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[12:43 p.m.] Q. And this is a gas price specialist, 
person from the gas industry? 

A. He is responsible for the fossil few 
prices, the fossil fuels. 

Q. Is he an economist or an oil and gas 
industry person? 

A. An economist. 

Q. Somebody who's worked in the oil and 
gas industry though. 

Ass -No. 

Q. Your gas price projection is 
essentially totally reliant on the judgments of this 
one person, isn't it? 

A. Yes, and reviewed by me and by the 
section head. 

We have, as you are probably aware, had 
Our gasS price peo eat ene reviewed by specifically 
one -—- well two, by two external experts, one U.S. and 
one Canadian, for the purposes of the thermal cost 
review. 

Q. But then you have changed them since 
then; haven't you? 

A. Not significantly. 

Ocwy Lh Seem tonwrecall: thatwyou shadiia 


number of $6.04, or something like that that you 
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reduced to $5.80; does that ring a bell, for the price 
25 years from now in real dollars? I am just doing 
this from memory, I don't have it in my notes. 

A. That's possible. If so, your memory 
is better than mine. But I would still say that 
calling a difference of 20 cents in current dollar 
price on a base like that is not a big change. 

Qo EB levsinet=that muchss ,.lthiseaboutero 
pergcentrofstoddy!siprice-Hisnee ree 

A. Well, I don't know. From the numbers 
you gave, it's 20/60ths -- or about 2/60ths, about 3 
per cent of the price in that period. 

Q. Do you supply your natural gas price 
forecast to the non-utility generation division for 
their non-utility generation plan? 

A.” -Yes:. 

QVevEs ee Haat to say that you have over 
forecast natural gas prices, the likely result is to 
depress the amount of economic non-utility generation? 
Is that something that you would expect to be the case? 

A. If the non-utility generation is gas 
fired, yes. 

MR. SHEPHERD: Mr. Chairman, I am 
planning to go into the area of uncertainty, which 


would be the rest of my questioning, perhaps we need a 
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breather before I start. 

THE CHAIRMAN: Would it be better to 
start after lunch, in your view? 

MR. SHEPHERD: I think it would be 
better, yes. 

THE CHAIRMAN: We will come back at 2:15. 

THE REGISTRAR: The hearing is a 
adjourned until 2:15. 
en=LUunGheonarecess#atv12:47, prem. 
aseOn, mesimingyatA2e 2 S-tpin. 

THE REGISTRAR: Please come to order. 
This hearing is again in session. 

| MR. SHEPHERD: Should we wait for Mr. 
Campbell? 

THE CHAIRMAN: Does the panel feel safe 
in going ahead without Mr. Campbell? 

MR. ROTHMAN: We feel safe enough, but we 
fear his wrath if we do. 

THE CHAIRMAN: Mr. Shepherd, perhaps you 
could give us an idea of what your time frame looks 
like at this moment. 

MR. SHEPHERD: Well, I do this with some 
trepidation, but I will centaur y be finished before 
the end of the day. I will not be finished by the 


break. 
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THE CHAIRMAN: Pardon me? 

MR. SHEPHERD: I will not be finished by 
the break.,,..but, lewibt certainly be finished by the end 
of the day. 

THE CHAIRMAN: Who is next on the list? 
I have next on the list Solar Energy Society, but then 
I guess, Mr. Thompson, you are the next after that at 
think the Solar Energy Society wasn't going to be here 
today. 

MR. THOMPSON: My understanding was that 
there had been a switch between the Solar Energy and 
somebody else. 

| THE CHAIRMAN: That was the NAN group, 
yes. 

MR. THOMPSON: If they are prepared to 
come on, Start today, I don't want to interrupt any 
other arrangements that have been made. 

THE CHAIRMAN: But you are prepared to go 
today, you are ready to go? 

MR. THOMPSON: Yes. Mine will be very 
brief, no more than five minutes, I hope. 

---Off the record discussion. 

THE CHAIRMAN: Why don't you go ahead, 

Mr. Shepherd. I think it would be all right to go 


ahead. 
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MR... SHEPHERD: «Ose Alleof.sthe fest: of 
this material is on your uncertainty methodology. I 
guess I would like to start by confirming, Mr. Burke, 
that the uncertainty band on the load forecast, you 
consider is very important, don't you, it's a very 
important component of your forecast? 

MR. BURKE: A. That the forecast is an 
arranged forecast is an important aspect of the plan 
and that planners should be taking into account the 
range of results possible for load in their planning. 

The quantification of uncertainty in 
terms of priorities within the load forecast department 
hard to rank in the sense that we certainly put a lot 
of effort into improving the median load forecast and 
the quantificationjofluncertaintynissximpontantyebut I 
would have difficulty prioritizing the two. 

Q. Mr. Rothman said at an earlier 
point - you don't need to turn this up unless you 
disagree with it - that the forecast is a median and a 
range; they are not separable. That is a fair 
assessment of the situation? 

A. Yes, and that's why I said, that we 
would like planners to work with the range forecast for 
planning purposes. 


OSe Andayourhaverinrc£fact isaidainivour 
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load forecast itself, the written document, the 
probability distribution associated with the median 
forecast is as important as its central tendency, 
correct? 

A. Yes. 

Q. But as you know, you do spend a lot 
more time working on the median than you do on the 
bandwidth? 

A. Yes, and I think that's appropriate. 

Ooo st amenoc™ peingreritical, el val 
interested in knowing why you would feel that's 
appropriate. 

A. Well, I think that quantifying 
uncertainty is a difficult task but in the end I think 
that there are going to be no absolute answers to this 
question. 

I think I began one of the chapters, the 
last section, rather, of Exhibit 10 with the phrase, 
"There will always be uncertainty about uncertainty." 
There is a limit to what you can do in quantifying 
long-term uncertainty, so that I think it deserves to 
be done as well as we can do it, but past a certain 
point I am not sure that it is worth in an infinite 
amount of resources. It certainly is a limited 


exercise. 
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Q. I guess I want to explore first how 
you actually find the function that you are using to 
estimate bandwidth. How do you come up with that 
formula? What is the process? 

A. Well, I think the process is quite 
well described in Exhibit.10, and in that, on ‘page 3, 
we have some objectives that we were looking for in 
developing the methodology. One of the objectives as 
far as the function is concerned is that because we 
want to take advantage of the information in our 
end-use and econometric forecasts that are used to 
derive the median forecast, that is we think that is 
the best median forecast oe: can make, we put a lot of 
effort into doing that as well as we can. 

So, it is, important that the load 
equation ae the median forecast well. It is not 
trying to forecast; it is simply trying to track the 
forecast we have already made through our modelling 
methodologies The reason we want it to track well is 
that if it did not track well, there would be 
distortions associated with then adjusting the results 
of the uncertainty bands that we developed with that 
equation to the median which we have generated using 
the end-use and econometric and judgment approach. 


So, thesfarsteprionity, tsrthat aitvtrack 
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the load forecast that we have made. And the second 
stage of the approach is to look to see how we can 
develop uncertainty bands for the explanatory 
variables. Well, in the case that we selected in 
Exhibit 10, there is only one variable, GDP, but there 
are other examples of equations and approaches to 
uncertainty given in Exhibit 10, so again, we would 
have had to have approaches for uncertainty for those 
other explanatory variables had we gone with an 
approach that had more than just GDP as the explanatory 
variable. 

So, the load equation has to have the 
property thateite@tracks wea And the basis for 
generating GDP I have talked about at several points in 
the hearing, we want an equation that could 
meahatatts Vesa produce for a wide range of time 
intervals a band that was judgmentally reasonable. 

So, we have tried to be as Simple as 
possible in these equations because the more variables 
you have the more complex it is. Although as Exhibit 
10 describes, we looked at an approach which had the 
price of electricity and the price of natural gas in an 
equation but also fitted fairly well, not ae well as 
this particular load equation with GDP and GDP squared. 


And, in fact, that other approach with gas prices and 
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electricity prices in the equation produced an even 
narrower band than the one we are getting with this 
approach, but it didn't fit well and that was one 
reason for doing it. 

- The other reason was we thought it would, 
in the end, be more controversial how we handled the 
uncertainty for prices for natural gas and electricity, 
and if we could get as good or better a tracking of our 
median forecast without introducing those variables, 
that, in a sense, avoided a perhaps debate over things 
that may not be that easy to resolve. As long as the 
equation tracks well in the period for which we have a 
forecast, that is to 2015, there really isn't the 
factors which cause it to have a decreasing elasticity, 
which is a property of our median forecast, come 
through in the determination of uncertainty, and 
really, that's what is important. 

Q. Mr. Burke, the process you use to get 
this, as I understand it - correct me if I am wrong - 
that you apply a number of mathematical techniques to 
the historical data to get a possible functional 
relationship, correct, and then you use trial and error 
to;rgetrone that efi ts eicwthatttaitr 2 

A. As you know, we have said we don't 


think forecasting long-term should be done with single 
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equation models for a variety of reasons. There really 
was a fairly restricted class of models that we could 
consider as long-term forecasting models that would 
have a chance to track the properties of our median 
forecast derived using a much more disaggregated 
analysis. So, it wasn't totally trial and error; there 
is a pretty limited set of candidates. 

Q. Once you have an equation you think 
looks like it might be right, you talked about fit; how 
is it that you determine that it is, in fact, the right 
equation? 

A. Well, for the purposes of this 
analysis, if you turn to page 103/104 of Exhibit 9, the 
1990 load forecast, it gives the amount of adjustment 
required to tune, you might say, the Single equation 
model to our median forecast, and very little 
adjustment is required to accomplish that. 

Exhibit 9, I believe, is the 1990 load 
forecast. 

Q. Page 103 you said? 

A. 104. So this equation -- 

Q. We have this numbering problem that 
we were talking about earlier. Is 106 in my copy? 


Asim SOEnY.S wees 
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P2230. Dems.) QO- That's okay* 

A. So, essentially, we're not looking 
here for a forecasting equation. We're looking for an 
equation that should have sound statistical properties, 
reasonably sound. But I don't think that is really 
even the major criterion. 

The essential feature is that it tracks 
the forecast in a fairly parsimonious way because 
certainly you could fit an equation that ultimately, 
perfectly replicated the forecast that in the process 
appeared as if there was no uncertainty left at all to 
explain. So, I think simplicity in design is 
important. We could have added variables aha variables 
and variables and reduced the residual uncertainty to 
the point where it was fairly small, but we didn't want 
to do that. We wanted to acer itT a simple equation 
and leave as much residual uncertainty and co-efficient 
error aS was reasonable for a long-term model. 

Q. Now, when you get an equation like 
this, you then test it; correct? 

A. To see whether it tracts the 
forecast? 

Q. Whether it tracks the forecast and 
whether if you used it in the past, it would have 


produced better results than what you were using in the 
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past. 

A. Yes, and for long-term forecasts an 
ex post test was done in the report by Dr. Cheng 
entitled, "Forecasting Ontario Hydro's Load Growth 
Uncertainty," which was filed along with at least 
Interrogatories 1.9.50 -- and I don't know whether 
there is one of yours that has it, too. 

Q. Yes. 

A. In the appendix to that document, he 
gives an ex post forecast performance result for his 
equation I think from about '76 through to about '87. 
And while it did not have the best fit of those 
equations, it -- that is over hestory in ane forecast 
Simulation, it performed better than the more dynamic 
short-term equations that we tend to use for the 
Short-term load forecast. | 

Q. Let's go to that 1976 test. When you 
take your new equation and you forecast it froma 
hypothetical 1976 start - all right - you take out all 
of the data past 1976 in terms of your calculation of 
your co-efficients; right? 

A. We're now talking about the just 
testing the equation itself? 

OJgsMes a) Yes: 


A. Yes. 
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Q. So you pretend that you only have the 


data up to 1976? 

A. (‘That's correct. 

Q. And you re-estimate your 
co-efficients based on the data that you have-- 

ASV yess 

QO. vetratvethe.tinie. GNow, inal formula 
like this you also have exogenous variables; right? 
It's driven by an exogenous variables? 

ALOwyves. 

Q. When you decide what -- you have to 
know the numbers for in this case GDP in order to 
project load? 

A. Yes. 

Q. You have to have some numbers for 
GDP. Did you use the actuals or did you use what you 
had forecast in 1976? 

A. ©To test the equation's forecasting 
performance? 

Opat Ves’. 

A. I'm not sure exactly, but I would 
assume, because it is the conventional approach, that 
if you're trying to test equations relative to each 
Other, that, in fact, you can't really do an absolute 


pure test, but you do take the actual values for GDP 
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and other explanatory variables, and really what you're 
testing is the model's ability not, the forecasting 
ability of those people who were forecasting exogenous 
variables. I also believe that we did not have, or we 
Still cannot obtain a GDP forecast from '76 to "89 that 
was made by our division to work with. 

Q. Now, when you look at forecasting 
error in addition in the error in the model, you also 
have error in the explanatory variable; right? 

A. Yes. 

Q. You're uncertainty is comprised of 
both? 

A. Yes. Well, I think we have to be 
careful whether we're talking now about forecasting or 
we're talking about trying to estimate uncertainty. 

Are you switching? 

QO. Wo. 

A. No. Okay. Well, to test forecast 
accuracy, it's very difficult to test forecast accuracy 
On an ex post basis without making some compromise 
about what was known and what wasn't known. 

And, really, if you play by sort “of. an 
equivalent approach; that is, if you don't give 
different sort of exogenous variables forecasts to 


different equations, you can really do a fair test of 
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the relative forecast performance of the equation, 
given a particular GDP forecast. That's all that 
really can be tested with that sort of approach. 

Q. So, you took out the component of the 
uncertainty that relates to the exogenous variables, 
didnijwteyou2eeThat\s noth m Yourmtest» imeN76? 

A. Well, I think, again you shouldn't 
use the word -- the word "uncertainty" isn't 
appropriate here. You're talking about potential 
forecast error now. You've asked me about a 
forecasting simulation from '76 to '88 or '89, and 
there is another source of error in the forecast 
associated with the error in the GDP forecast that 
would have been made at that piont. 

And, yes, in what was done in Dr. Cheng's 
report, the error in GDP was not taken into absédnv. 

Now, we did just, just so I understand 
we're on the same wavelength here, we did provide you 
with information on another ex post simulation for 
uncertainty purposes in an interrogatory response, and 
that was done differently. There we were concerned 
about having an understanding of GDP uncertainty in the 
future. 

And so we estimated what we would have 


derived using the methodology we've got here in Exhibit 
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10 for the GDP band, had we been doing this in 1976. 
That's a different question than the forecasting side 
Oftchits 

Q. Now, Mr. Burke, as I understand your 
previous testimony - I'm not sure whether it was you or 
Mr. Rothman - your 1976 test showed approximately 5 per 
cent load growth, as opposed to the 7 per cent that you 
had forecast at the time. Is that about Fao grine 

A. Well, I think we could turn up the 
interrogatory, but there were two cases presented. 
There was the adjusted and the unadjusted case, I think 
it was referred to. 

The equation produced a value for GDP and 
hence a value for load, that is, the GDP equatronze that 
result in load growth of around 5 per center bAndtthe 
load forecast of the day, was 7 percent. And #7 we 
had applied the methodology that we use now, we would 
have adjusted -- 

Pirst of all, our equation wouldn't have 
tracked the forecast very well, and we might at the 
time have decided to reject that equation because it 
certainly didn't satisfy the test that it was producing 
at that time a load forecast similar to the one that 
was being made at that time. 


However, given that, we can't change 
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everythangsand stillscalltitcattestv® We®képt’ with the 
Same specification for the GDP equation, load equation. 

The difference between the 5 per cent 
that the pure model simulation would derive and the 7 
per cent that was the forecast of the day, made the 
difference between whether or not the bands captured 
the actual results between '76 and ‘88 or '89. 

Q. Of course isn't it true that the 
Forecast of the day also included some error in the 
exogenous variables because you didn't redo that 
forecast of the day having already known what was the 
results would be in the exogenous variables, did you? 

A. My recollection of our discussion of 
how forecasting was done in 1976 is that - it's on the 
record here - we certainly did not have end use and 
econometric models. It's not clear exactly what was 
the basis of the forecast done in 1976 for the 
long-term. 

We know how the first five years was 
done, but I don't know whether there was an equation 
that we could have -- I don't think there is an 
equation that we could say was the basis of the 
forecast, the load forecast, that was made in '76. 

Q. Isn't it true that the forecasts you 


did in 1976 would include in it some uncertainty or 
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ultimately some error resulting from those exogenous 
variables - whether it's population, whether it's GDP, 
whatever? 

Ao he sfGnedcas t2hoves ; 

Q. It would include that? When you 
forecast 7 per cent, it would be wrong partly because 
your model wasn't that good and it would be wrong 
partly because you didn't get GDP right because you 
didn't know what GDP was going to be? 

A. That's possible, but it's also 
conceivable that the long-term forecast was simply run 
out at an average rate over historical time. I do not 
know what was the basis of the 1976 load forecast wh 
Cannot say were there exongenous available errors and 
SO on at the time. We do not have that sort of 
information, unfortunately. 

Q. Isn't it fair to conclude that when 
you're comparing the 7 per cent that you had then to 
the 5 per cent that this model produces, you're 
comparing apples to oranges -- there's no direct 
comparison between the two numbers? 

A. No, I don't think that is the case. 
I think that it was as reasonable a test as can be made 
and I think to reject the sort of test we did would 


essentially suggest you cannot do ex post testing 
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meaningfully. 

Q. Am I right in understanding that the 
7 per cent number that was -- 

A. Actually, something is occurring to 
me. 

Q. Go ahead. 

A. I think you're arguing that load 
forecast, the 7 per cent number had some uncertainty in 
explanatory variables in it, and I have said that for 
the purpose of testing uncertainty, that is, whether or 
not the 80 per cent band width produced in '76 captured 
the actuals, that we did have uncertainty -- I just 
want to be pia Ener we did look at uncertainty in GDP 
in performing that test. It was a complete simulation 
of the system that we have. 

| So, it's not like the simulation had no 
GDP uncertainty and implicity the other simulation did. 
The difference is in the median forecast, not in the 
treatment of uncertainty. The median forecast 
generated by this two-equation system left to its own 
devices using the same approve we do today was a 5 per 
cent forecast, but the treatment of uncertainty is 
parallel to what we do today. 

It's important to make this distinction 


between forecasting with an equation and looking at the 
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error in the forecast and using an equation to estimate 
uncertainty about a forecast trend line which you 
derived, in our case, some other way. 

Q. Maybe this is totally simplistic, but 
I guess I've always understood that uncertainty that 
you forecast today becomes error in the future. They 
are two parts of the same thing. 

A. Once you have the actual, you can 
calculate an error from your forecast. The Simulation 
experiment that we were talking about was one in which 
an equation left to its own devices produced 5 per 


cent, did not track the forecast of the day very well. 
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[2349 ben.i When we adjusted it up to the forecast of 
7 per cent, the band did not include the result that 
occurred. However, the observation was made that 
perhaps if the equation was almost providing a better 
forecast, and one more in tune with the sort of 
forecasts we would make with today's methodology back 
then, it might in fact -- the equation's result may be 
more indicative of what we would do today, given that 
it is 12 or 15 years later and the forecasting 
methodology has evolved and we are now looking at 
Structural shifts and all kinds of things that alter 
the load forecast from a steady 7 per cent trend rate. 

Q. Let me go on to I guess what may be a 
related point. 

In your testimony on April 25th - I won't 
give you the transcript reference unless you ask for 
it; I think you will remember - you were somewhat 
Surprised and dismayed at the MEA's revelation that 
your load forecast bandwidth was narrower than your GDP 
bandwidth. Is that now solved by the way by your 
correction last Wednesday? 

A. No. The relationship between the GDP 
bandwidth and the load bandwidth, I think what I was 
dismayed about was the premise that Mr. Mark felt he 


had gotten me to agree to. What I had agreed to was 
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that if you simulate the load equation and you include 
errors in the co-efficients and you include errors in 
the residuals, you will get a wider band than if you 
Simulate that equation with GDP alone. 

But, there was a presumption that the 
equation translated a GDP on Mr. Mark's part, and I 
under the pressure of cross-examination - I think 
that's what was confusing me at the time - that that 
GDP band translates one to one into a load band, but 
the point of the equation is that tb) doesn“; "thath iby 
the time you get to 2010, which is the point in the 
future that all Mr. Mark's cross-examination referred 
to, the elasticity between GDP and load is 0.85 or so. 

And that suggests for any given value of 
GDP, the corresponding value for load is about 85 per 
cent as much. It shrinks the band. It Says that as 
electricity intensity decreases, Changes in GDP 
uncertainty have less and less impact on load 
uncertainty if you don't take into account the 


co-efficient and residual errors. 


Q. So, this is the correction you wanted 


to make on April 29th? This is the addition-- 
A. Yes. 
Q.  --you wanted to make on April 29? 


A. Yes. And I can point you to the 
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places. But I can see how the misunderstanding arose 
and it certainly confused me at the time. I will 
certainly admit to that. 

But the issue is the assumption that 
essentially these uncertainties are additive; that you 
have got GDP uncertainty and you add co-efficient 
uncertainty and residual uncertainty. That is not what 
is going on here. We have an equation that transforms 
GDP into load, and we also have to take into account 
multiplicatively the uncertainty in the co-efficient 
and uncertainty in GDP and additively the uncertainty 
in the residual. 

Q. Does that mean Bhat itiisethenvfair 
to conclude that it is quite reasonable to expect the 
load forecast bandwidth to be narrower than the GDP 
bandwidth in percentages? 

A. Well, it depends ultimately on the 
relationship, what the output elasticity is in your 
forecast in the median case, and it depends on how much 
uncertainty, which is adding to the range generated by 
the equation comes from the co-efficients and the 
residual. 

If in estimating this equation, it is 
apparent that that trend to declining intensity is 


known with a high degree of confidence, the uncertainty 
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in that co-efficient and in the residuals may not be 
enough to offset the 85 per cent shrinkage by 2010 in 
the load band. 

But, it could have been the case that 
historically this was a very volatile relationship and 
we would have found ourselves in a situation where the 
load band could have been greater than the GDP band. 

BEfectivel yds I said, simulating with 
GDP alone, you are going to get less, a narrower band 
for load than for GDP given a forecast of declining 
elasticity, but adding in the errors on the 
co-efficient and the residual will widen that band. 
Whether it widens it to the point Raat iti seaetual ly 
greater than the GDP band is a matter of empirical 
determination. 

Ove ik Ve it then we shouldn't take 
anything from your surprise and dismay on the 25th; 
that that was just -- 

A. I was confused as how I had gotten 
into the situation where the methodology which I, in 
reviewing the transcript, described correctly. but 
perhaps not very convincingly, because I was describing 
it in terms of the negative co-efficient on the 
x-Squared term and how that had the effect of narrowing 


the band. It is certainly a lot clearer if I tCaae in 
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terms of the overall elasticity in our median forecast 
which is replicated by this equation. 

Q. So, in fact, using your model then, 
it's perfectly acceptable in the appropriate 
circumstances to have a narrower bandwidth for load 
than it is for GDP, right-- 

Are? es',) that isivcorrect’: 

Oemirawith¥ithiicmelasticity? 

Okay. Last Monday when you were being 
cross-examined by the Coalition, I believe, there was 
some discussion about the standard deviation for 
population? 

Ass Yes. 

Q. How you would have done it. 

And I am totally confused on that. 

Doi understand correctly that the 
standard deviation for population controls your 
population uncertainty band, the uncertainty of the 
population number? 

A. Controls which band? 

Q. The uncertainty band or the 
distribution, if you like, for the population number. 

A. Yes. The standard deviation of the 
history of compound growth rates of a population of a 


particular period, that is, 17 years, 25 years, that 
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Standard deviation is used to generate” a *——* that 
Standard deviation is used to a generate a distribution 
for population uncertainty for that number of periods 
into the future. 

Q. Standard deviation is in fact the 
tool you use to create probability distribution system? 

A. Yes, that's right. As we point out, 
this is the one statistical assumption we make in the 
approach we have called stochastic Simulation; that is, 
we had to assume something about that distribution and 
we assumed it was normal. 

For the remainder of the exercise we 
don't impose any particular distributional assumptions 
on the standard deviations of the explanatory 
variables. 

Ov 1GANSiealsouright®in assuming the 
changes to the uncertainty band for a population, let's 
Say have created an 80 per cent uncertainty band for 
population, if you change that, that will have a direct 
and concommitant impact on your uncertainty band for 
load; correct? 

A. It does only if you don't do what we 
do which is to judgmentally assess whether the GDP band 
that you derive makes sense; that is, we do a sanity 


check on the GDP band and really it's very important to 
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uscthataweiarehinethe rright®ballpark for «GDP. Afterall 
we are not doing this as a once-through operation; we 
are doing this as a search for a GDP band to put into 
our load equation. 

And effectively if we were to come up 
with a GDP band that didn't make sense, we would scale 
by some other factor. Clearly we have scaled already 
to derive the GDP band we have. That reflects a 
judgment that without scaling we would get a 
nonsensical GDP band and judgment in the eyes of the 
people who produce our GDP forecast and I guess 
relative to this survey. 

So that, while it is mathematically 
correct that if you string them together, don't touch 
anything, you would get a fairly direct relationship 
between the population standard error ana the load 
Standard error. That is not the process we follow. 
The process is laid out quite clearly. We generate a 
population band to get a GDP band. It is the way we 
get our GDP band. The GDP band has to be reasonable. 

And I think I have testified to that at 
several points in this hearing so far that the survey 
that we have done - and right in my direct evidence - 
the survey we do as a reasonableness check is quite 


important. 
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We also have other benchmarks. We look 
at the history of GDP and its growth rates without 
doing an equation at all. As you pointed out, Mr. 
Shepherd, on Thursday, the search for how you treat 
that one driving variable, that last one that drives 
everything has to be done carefully and there are 
judgments entailed, and so we have done it carefully. 

And it's not mechanistic, we take 
something to population and run it Erght -thfoeugh..+ Or 
else we wouldn't have scaled in the first place and we 
would have ended up with a GDP band that no one 
believes represents an 80 per cent confidence interval 
for GDP. | 

Q. So, the sole way you test it then is 
to see whether you like the answer? I am not being 
facetious. In essence? 

A. As I have said, there is certainly a 
heavy judgmental element to the GDP band itself;* And 
the benchmarks we had and are described in SecET On] Saof 
Exhibit 10 -- well the benchmark that is described in 
section 5 of Exhibit 10 is the Survey of the external 
economic forecast advisory committee. 

And as it turns out, our GDP band 
generated by the model is wider than that survey; i +a 


think we accepted the extent to which it is wider than 
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the survey as something prudent, but we don't want to 

pick the narrowest numbers around. But effectively we 
didn't want it to be much wider than it is because, in 
fact, we are starting to depart significantly from the 
public, the opinion of experts in forecasting GDP. 

QO. “Let»me*come back® to that in a second. 

Let me just nail down this one point. If 
you double the standard error for population, you 
double the bandwidth for load; don't you? 

A. I think I have been quite clear that 
if you-do -it mechanistically; ’that is; /if\ you run’ the 
two equations together and you don't do anything to 
assess the GDP band on the way, that is what the 
mathematics tells you. But that is not what we do. 

I think in an interrogatory response on 
that question -- I don't know whether I put enough 
caveats into the response but certainly what I was 
giving you there was the result of mechanistically 
doubling the standard error. Not would we, in fact, do 
that? 

Q. I guess I'm having a little trouble 
understanding that. I would like you to look at page 
LOS+O8 Exhibit 97Myourtpagesl030" This isvactually out 
of the OEB materials, so the numbering is correct. 


THE CHAIRMAN: What is everyone else's 
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page? 

MR. SHEPHERD: It would probably be 105, 
yes.» st teynsectionmg es of£ithat interrogatory. 

Q. And this looks to me like you say you 
do Monte Carlo simulations at each of the two stages of 
your process but it doesn't say here that you then 
change them if you don't like the answers. 

MR. BURKE: A. That's because we already 
know that this scaling methodology yields a GDP band of 
the right order. That was determined several years ago 
and is still the case and that's why we scale by the 
amount we scale. 

It certainly must strike people that it's 
somewhat arbitrary to scale the population standard 
deviations. The judgment was made and we point out 
that there is a judgment on, I think it's page 10, at 
the bottom of Exhibit 10, that the confidence shown in 
the population forecast is one of the judgmental 
elements in determining the uncertainty band for the 
load forecast, and effectively that which is at the 


point at which we do the scaling. 
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(1311004 ps min) If we did not do that scaling we would 
not produce a GDP band that was remotely close to our 
Survey results. So we are obliged to, in fact, do that 
sort ofiscaling. 

Q. I am going to come back to the survey 
results in a second, but when you scale the standard 
deviation, which you have done, right, is there a 
technical basis for doing that? Is there a technical 
reason why that's necessary aside from the fact that 
you want to get certain types of results? 

A. No. It's done because we want to 
achieve a GDP band that is reasonable. 

As I have said before, the alternative to 
this is a totally judgmental approach. I am not 
familiar with anybody, particularly, who tries to model 
GDP Rae ei wees this sort of exercise. We did it 
for reasons that I have described, that we have to 
produce a year by year uncertainty band and we wanted 
to be able to do it fairly mechanically once we had a 
system that was calibrated into the right range, and 
this sort of assumption was what it took. 

Q. It sounds to me like what you are 
Saying is that you take a look at what the other 
forecasters tell you, and that is in effect your GDP 


band, and then you find an equation that matches that. 
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Avy Wellowitits mothas fart ftom’ that as it 
might seem, no, but it's not that we see what other 
people forecast. 

We specifically ask them what the 10 per 
cent and 90 per cent points of their long-term view of 
Ontario GDP was. So, we are asking them, without 
giving them any feedback on what other people are 
Saying, to give us a long-term forecast bandwidth for 
GDP, and that is used as a reasonable indicator bute it 
isn't slavishly followed. 

Our own forecasting people look at that 
and, in the end, we have come with a factor uSing a 
wider bandwidth then that Survey result indicates. 

Q. I am not sure I understand whether 
you use the results of your model or the results of the 
external forecasters or some combination. 

A. Well, given that we have a certain 
amount of control over the results of the medet;tit's, 
you might say, some combination, because if we didn't 
scale the model, it would not produce a bandwidth that 
we were happy with. And if we use the survey results 
we would be narrower than we are now, which concerns 
Our own forecasters because they are not sure they can 
really deliver a forecast that accurately. 


Q. Now, when you use the smaller 
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standard deviation, isn't one of the results of that 
that if your estimate of population growth, which is 
the exogenous variable was zero, then mathematically 
the result would be you would be 100 per cent certain 
of that answer? 

A. Yes, this is designed to achieve a 
certain reasonable result. Obviously if we had zero 
population growth we wouldn't use the same approach. 

Omntso ido ritatakeisrtat hens. that Dovonly 
works at one per cent, it's only correct at one per 
cent or reliable at one per cent? 

A. This particular mathematical way is 
valid in the close neighborhood of the forecasts we are 
working with. And if the population forecast started 
to depart significantly from the population forecast 
and the region around it that we were working with, we 
would have to do something different. 

This is a device to achieve a reasonable 
GDP bandwidth that can be mechanistically simulated for 
the sorts of purposes that we require GDP bandwidths 
for. It does not claim to be the only way to do it or 
the correct way to do it in all circumstances. 

Q. The way it's designed, it works best 
at one per cent of your own, of whatever your median 


ren ght 
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A. Well, it works in the range. I am 
not sure whether it's only at one per cent, but it's 
certainly, as you get closer to zero, clearly we 
wouldn't do the same thing. 

Q. And presumably as you get closer to 
two or to three or to four you would also have some 
concerns about how it worked? 

A. Well, it would depend on where that 


took the GDP band. That is, if we were dealing with 3 


or 4 per cent population growth in Ontario for the next 


25 years, we would be having GDP forecasts that are, 
well, typically one-and-a-half to two per cent higher 


than that, and frankly, at that point I would ask Mr. 


Rothman what he was doing with his demographic forecast 


because it wouldn't pass, even the demographic forecast 


wouldn't pass the reasonableness test, let alone the 
GDP forecast that results from it. 

Q. Clear something up then for me. It 
sounds like as you get farther away from your median, 
this model gets more and more wrong. At one per cent 
it's pretty good, at zero per cent it's totally crazy. 
Somewhere in between it gets worse and worse. 

A. Well, we had this interesting 


discussion about the interplay of judgment and 


quantitative techniques on Thursday. This is certainly 
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a case where if you take a purely quantitative 
approach, you are going to get yourself into trouble 
unless you apply some judgment. And this approach does 
not claim to handle all circumstances. 

It's objective is to get a reasonable GDP 
band and one that we can then produce all the glorious 
numbers that are required for planning purposes ina 
great amount of detail but is in a reasonable range. 

If the band is not in a reasonable range, we would 
change the scaling factor. So that if we started to 
produce absurd results, we would do something 
different. 

Q. Isn't the notion or the essence of 
bandwidth the notion of variances from the mean? Isn't 
that what you are doing in bandwidth, is calculating 
variations from a mean or median? 

A. What I am trying to do is to estimate 
a GDP bandwidth and this is the way we have gone about 
doing it. That is my objective. 

As we indicated, it's a challenging 
exercise. We could have done it judgmentally, we could 
have done it purely with a survey. Because we needed 
to do it in much more greater detail in all kinds of 
cases that don't lend themselves well to surveys, we 


chose to do it this way. But, consider the survey, 
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consider the history of GDP but itself; that is why go 
to all this trouble to model GDP if you can just use 
the history of GDP by itself. That turned out to be 
too narrow a band. Nobody found that credible because 
it's much narrower than the band we get from the survey 
and even narrower still than the band we get*using’ this 
equation. 

Q. I guess I am having a hard time 
understanding why you have got this model that as you 
move to the lower probability futures, if you like, it 
gets more and more wrong, and yet -- I am not sure why 
you would have a model at all if what you are relying 
on is what the other forecasters Say. 

A. The scaling factor is not something 
that is simulated. The scaling factor is set once and 
for ‘all. INSo, lit’ swicumaetiie during simulation somehow 
we end up in a situation where the uncertainty that 
measures the scaling factor is set for the median. 
That's it. It doesn't change. It is not Simulated. 
So, that aspect is not a function of dispersion about 
the median of population. 

The only thing that is a function of 
dispersion about the median of population as far as it 
pertains to load -- well, no, that's if you carry 


through directly without looking at GDP. 
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Q. Do I understand correctly then that 
the way you got to this scaling factor is you said how 
wide do I want this bandwidth, and whatever the right 
number was, that's what you used? 

A. The bandwidth for GDP is ultimately 
judgmental. It is a range forecast against various 
sort of reasonableness tests of what a GDP band should 
look like, yes. 

Q. So your answer to my question is yes? 

A. Well, not exactly what you said, what 
I -said, 

Q. All right. Let's go to the the next 
issue with respect to your model structure. I am going 
POF aSkBVGUREOLLULNAUpVEXAIDICt 97% 

THE CHAIRMAN: 97? 

MR. SHEPHERD: 97. It's entitled "Notes 
on Ontario Hydro's Forecasting Activities", the author 
is Dr. Christopher Chapman. 

THE CHAIRMAN: Okay. I have got it 
somewhere. I know I saw it. 

MR. «BURKE: “4Yes, “I have: that: 

MR. SHEPHERD: Q. It might be convenient 
Vi, syou yalsopturniup Exhibit 1L30kwhtch is’ Drs de Bever's 
short paper on the same issue. 


MR. BURKE: A. Yes, I have both of 
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those. 

Q. Dr. Chapman has, in a very polite 
scientific way, raised serious doubts about how your 
model is structured in fact, your uncertainty model is 
structured in fact. He says it's just unsound. The 
math is wrong. Dr. de Bever has referred to it as 
structural unsound, called it functionally unsuitable 
for the use to which you are PuUtCangiyi t< 

Maybe we could short circuit this a 
little bit by askingu yourypsin light of these comments, 
do you agree with those criticisms? 

A. No, I think they are invalid. 

Q. Can you tell us why? 

A. ©There are twospoints that#Iethink. Dr. 
Chapman wants to make, one concerns the matter we have 
been discussing up to now, which is the scaling of the 
Standard error of population. And on page 4 of his 
material he generates a bandwidth for load using the 
Standard error that he perceives to be more 
appropriate. 

Implicit in this standard error is a 
growth rate, a GDP band at the 80 per cent level that 
ranges from 1 per cent to 6 per cent approximately. 
Our survey indicates that the results are in the range 


perhaps 1.7 to 3.6 of that order for a 10-year period. 
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In fact, the 1990 survey to 2000 indicates, I believe - 
Liwbllngustsburn . it up there a27e3 § 

THE CHAIRMAN: Are you reading from his 
paper? Those figures don't seem to leap out at me from 
page 4. 

MR. BURKE: No. On page 4 what he has is 
a load band. 

MS. PATTERSON: This is Chapman? 

THE CHAIRMAN: Where is that? 

MR.- SHEPHERD: Exhibit 97. 

THE CHAIRMAN: Yes, load band. Yes, I 
see that. 

MR. BURKE: I am suggesting that because 
Dr. Chapman did not do any sanity checks on his GDP 
band on the way, he has implicitly generated an 80 per 
cent GDP band, that is 1 per cent to 6 per cent for the 
next 10 years, and that our survey of experts done 
Llasteas 

THE CHAIRMAN: Those are calculations you 
have made from this? 

MR. BURKE: Yes, from these numbers, 

Cha tsar taht, 

THE CHAIRMAN: That's what I am having 

trouble following. 


MR. BURKE: I have used the upper and 
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lower levels, looked at the growth rates versus '89, 
made the simple assumption that roughly speaking load 
band would be slightly wider than the GDP band and 
said, for the very reasons that we go not use the raw 
Standard error, Dr. Chapman has produced a result that 
is clearly implausible. He has produced a band that is 
much much wider than expert Opinion of an 80 per cent 
band for GDP. 2.3 to 3.6 is what our external load 
forecast advisory people said was reasonable. 

THE CHAIRMAN: You are going pretty ‘fast. 
2s a) GE OaaserGs 

MR. BURKE: Is an 80 per cent confidence 
band for GDP according to a May 1990 external economic 
advisory committee Survey of its fifteen members. 

THE CHAIRMAN: What do you say Dre 
Chapman's is? 

MR. BURKE: Roughly 1 per cent to 6 per 
cent. 

THE CHAIRMAN: 1 to 6. 

MR. BURKE: Yes, much much wider. 

And I am making the statement that the 
reason we scaled was to avoid such a wide band in the 
first place. We obviously knew what we would get if we 
did not scale and we rejected that as unreasonable. 


MR. SHEPHERD: Q. Let me pursue that for 
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a moment, Mr. Burke. Your original bandwidth was 
14,000 megawatts, right, in capacity? 

MR. BURKE: A. which original bandwidth? 

CG. Thirteen-nine? Your 1990 bandwidth 
was thirteen-nine? 

A. For when? 

On FOr 2015. °SOrry, sais sistec000, 1s 
aK; wg 

Now, you have in fact changed yours, 
right? You changed your load forecast bandwidth 
substantially; haven't you? 

A. Yes, I have, yes, because of some 
properties beyond 2009. It doesn't change before 2009. 

Q. So, does that mean that you didn't do 
a sanity check either? 

A. I have admitted to my embarrassment 
on my stand that I produced a result that I 
judgmentally would not wish to have produced and 
therefore I have corrected it. 

Q. Have you gone back to the implicit 
GDP that would underlie that -- 

A. Yes, I have that. 

Q. And what is the new GDP band? 

A. “The=new GDP band to 2015 now -—- 


Is that what you are interested in? 
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Q. It doesn't matter. 

Ah SUSE aT Cots He percents 

Q. As compared to what? What was your 
previous one? 

Ay" Two-tows. 3) “so Par erre abpoue-d. 3 
added and 0.3 subtracted on either end of the 


distribution. 


Parr & Associates Reporting, Inc. 


24 


25 


Rothman, Burke,Buja-Bijunas 2145 
cr ex (Shepherd) 


Pero eos De] This is relative to several per cent 
difference between ourselves and Dr. Chapman -- 

Q. Yes. Dr. Chapman has argued that. 

MR. B. CAMPBELL: I'm not sure Mr. Burke 
is finished his answer. I had down that he was going 
to speak to two matters. I'm not clear whether he 
spoke to both of them. 

MR. SHEPHERD: I'm apologize. I lost 
track. 

MR. B. CAMPBELL: Okay. 

MR. SHEPHERD: Go ahead, please. 

MR. BURKE: Thank you, Mr. Campbell. The 
second point that Dr. Chapman wants to make concerns 
the forecasting properties of a system we don't use for 
forecasting. We have two equations. We use one to. 
develop a GDP band and we use the second to translate 
the GDP band into load. 

And so what Dr. Chapman demonstrates is 
that if you extrapolate quite extensively beyond the 
range of our forecast both in terms of time and in 
terms of the magnitudes of the driving variables, you 
can result in what Dr. de Bever has shown as a 
knife-edge property to the forecasting function, which 
would, if we were about 25 years from now doing this 


same exercise, result in the fact probably that that 
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equation would no longer track the median forecast that 
we prepare using econometric and end use methods. And 
we would reject the equation at that point. 

But I'd like to emphasize it would take 
roughly that long. If you turn up Dr. de Bever's 
evidence, he produces a very useful plete atuthe last 
page which -- 

MR. B. CAMPBELL: Sorry. What's the 
exhibit number? 

THE CHAIRMAN: 130. 

MR. BURKE: 130. Which shows that -- 
just to put things into context, the population of 
Ontario is something less than 10-million today, and 
we're forecasting in our median case a population of 
12.6-Mi Lion: 

If we use our standard error, the upper 
bound corresponds to 13.4-million. Sorry, about 
14-million. If we use Dr. Chapman's upper bound, I 
think we're around 15-million. 

But that means that the only point that 
really is relevant about this analysis is the extent to 
which, in simulating large values of population on the 
tails of the distribution in 2015, are we likely to run 
into values that are so large that they go beyond the 


knife edge here. 
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And so our simulation of the band width 
for 2015 is somehow influenced by the properties of 
this equation. And my assertion is it is not, that the 
knife-edge properties were not going to run into for 
about another 25 years. Population grows at about a 
million a decade in Ontario. The knife-edge is at 
20-million. We could properly be simulating 25 years 
from now and at the 80 per cent or 90 per cent points, 
not have values that are beyond the knife edge. 

Essentially, I disagree with Dr. 
Chapman's view that because - and Dr. de Bever's - that 
because this equation, which is a quadratic, has 
undesirable properties once the population exceeds 20 
million, that is invalid it to use%it’the fage of 13 
million. 

There is a lot of experience of using 
quadratic equations to approximate things in a useful 
range. This exercise is one of extrapolation well 
beyond the useful range that we're putting this 
equation to, sandesl Inu litt omvoultthat. ve Cin /facte we 
ever got to that point, it would no longer track the 
load forecast. And so it would be rejected as the 
equation to use to derive the band width. 

We have other we equations that don't 


have the quadratic property. They result in similar 
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band widths. So, I feel any results are robust in that 
sense. It's not a function of the equation type. I 
think this is quantitative analysis without judgment. 

MR. SHEPHERD: Q. I take it then that 
you're saying that sooner or later your equation is; in 
fact, wrong; it's just that numbers that it has to be 
wrong at are not within the range that you're 
expecting? 

MRS BURKE! sag Ives. 

Q. Now, when you use an equation like 
this, does it sort of suddenly become wrong or does -- 

A. Well, I think you can see the point 
at which it suddenly becomes wrong. It's quite clear 
in’ thisepieture: 

THE CHAIRMAN: Again, which picture is it 
you're looking at? 

MR. BURKE: It's the third page of Dr. de 
Bever's Exhibit 130. It shows that as population gets 
to about 20-million, it has some intuitively 
undesirable impacts on the load results, and -- but 
that in the range 13-million to -- I think the highest 
number we'd really run into is 16-million or so ina 
Simulation like the one we're doing for the year 2015. 
This equation is very linear. There isn't much that's 


happening that's particularly exciting. 
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THE CHAIRMAN: Again, just to be 
absolutely clear, is it graph No. 1 you're looking at? 
MR. SHEPHERD: Graph No. 2? 

ME eoURNGs “NO. I CINK ics fumoer 2° 

THE CHAIRMAN: Graph No. 2? 

MR. BURKE: TEp.) =. mM Suggesting. that: in 
thexregqiom-of-13-mibrlion;, plus -2="or 3-million, the 
equation is extremely linear, unexciting, doesn't do 
anything unstable and is quite adequate for the purpose 
COmweLcn 2rteils put, 

MR. SHEPHERD: Q. Well, if what you want 
is an equation that draws a straight line, why does 
your equation draw a curve line? I don't understand 
that. 

MR. BURKE: A. Because it has to track a 
EFOrecastethatenasvavdeclining™elacticity for load. It 
is a variable elasticity model. The variability comes 
in through the second term on GDP-squared, and that is 
what causes this model to actually replicate our load 
forecast where most single equation models would not. 

And in the region to 2015 that we're 
working with, it does it extremely well, as I pointed 
out on page 106, I guess, of the 1990 load forecasts. 
And that is what we intended to do, and it performs 


very well in the region that it is supposed to operate 
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Laity 

Q. So, the only criteria then that you 
would decide tht you like this equation is that it 
tracks your median load forecasts, which you've done 
more comprehensively? 

A. Parsimoniously in terms of variables 
that we can try to estimate the uncertainty of; that 
is, you -- in estimating uncertainty, one has the 
problem of also estimating the uncertainty in 
explanatory variables. 

SO, parsimony is a virtue, and we've 
picked the major driver. We've picked the only 
equation with the major driver that tracks the load 
forecastlie ein SGact, the class of possible candidates 
for this equation is very small, and we can get 
equations that are nearly as good with electricity 
price and gas prices in them, and they are given in 
Exhibit 10. And also in the background report, we have 
a background report on the Monte Carlo simulation with 
another one with that equation with price and price 
variables in it. 

The price variables have the effect of 
Causing the forecast to show decreasing elasticity over 
time, and they produce a band which is even narrower 


than the band that we have with this approach. So we 
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rejected it as too narrow. 

©. .I guess. the difficulty I*m*having is 
that you say, "Take a look at population within the 
range that you're expecting over the forecast period," 
and it looks to me like a straight line. It looks to 
me like, in fact, some of your variables are offsetting 
other variables to produce ultimately a straight line. 
isnt. that about. night? 

A. Well, I think. this is one of these 
things where magnifying the scale wouldn't hurt. But 
ii eSaNOL, aysiraighted ine eamtycurves slighbiys' eit 
curves the amount of our forecast, which is 15 per cent 
reduction over 25 years. 

QO. So the nice thing about this equation 
then is that it produces a reliable result. It does 
what you want it to do, produces a result that you 
like? 

A. I think we're quite clear that what 
the objectives are of the process of estimating the 
uncertainty associated with the median load forecast. 
It is important that the equation track the median 
forecast. It does that. It's important that it do it 
parsimoniously. It does that. 

We then have to estimate GDP uncertainty. 


We do that separately. We don't run the two together 
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just to see what numbers we can generate with extreme 
values. 

The range of the values here for which 
Stability exists is well beyond what we're likely to 
run into in Ontario in the year 2015. As I Say, 1 
think I could use this for 25 years and not run into a 
problem. 

I doubt very much I will use the same 
approach for the next 25 years, but this certainly is a 
test of a quadratic well outside the range that is 
relevant for the purpose of this hearing. 

Q. And if it you did use it: for the next 
25 years, could we expect it to produce the same sort 
of bandwidth that it produced last year? 

Ano aWed ly ra— 

Q. And to be just as wrong? 

A. ~ Pardon? 

Q. And to be just as wrong? 

A. I don't understand your point. 

Os Welbe=- 

A. What's wrong? 

Q. Didn't this equation produce a 
bandwidth of 13,900 megawatts that you told us last 
Wednesday was totally wrong? Isn't that right? 


A. What I said was that it had a 
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properctyra™ beyond” 2009 "that "rt arrowed, and, 
effectively, it was the GDP band that was narrowing, 
and, effectively, from our perspective, we didn't scale 
it quite right to produce a GDP band that is 
judgmentally reasonable. That should have been 
corrected. I have corrected it. 

It results in a GDP band which is wider 
than the survey throughout the period instead of 
converging to the survey by the end of the period. 

I'm now satisfied that the GDP band is 
reasonable. It produces a load band that is 
reasonable. I have not changed the load equation. 

MR. SHEPHERD: Mr. Chairman, this may be 
an appropriate time to take a break. 

THE CHAIRMAN: Right. After the break. 

THE REGISTRAR: 15 minute break. 


MoTReECess Sate 3250 ‘perm. 
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rie OO: x ABM Lids a te iSie-47) sas 

THE REGISTRAR: This hearing is again in 
session. Please be seated. 

THE CHAIRMAN: Mr. Shepherd. 

MR. SHEPHERD: Mr. Chairman, my experts 
this morning predicted that I would not be able to 
finish by the end of the day. I think at this point 16 
is probably nip and tuck. I am Going, toutny. 

THE CHAIRMAN: All right. 

MR. SHEPHERD: Q. Let me follow up the 
last point just with a couple of -- 

THE CHAIRMAN: I think, I'm not the only 
one, we are having a little ok celia atin ies you this 
afternoon. I don't know whether your mike is working 
Or if you are too far away from it. 

MR. SHEPHERD: I am probably mumbling, 
Mr. Chairman. Especially after that last interchange, 
I may be mumbling more. 

Q. Mr. Burke, the problem you had last 
Wednesday was from the year 2009 Onwards; right? There 
was no problem in the numbers up to the year 2009? 

MR. BURKE: A. That's correct. 

Q. But it is the same equation that 
projects the bandwidth for each year; doesnt it? 


A. The GDP equation is the same for each 
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year; and once the GDP bandwidth is determined, the 
load equation is certainly the same. I guess the issue 
is: Did the GDP band satisfy a reasonableness test? 
And one of the criteria which seems to say it did not 
was that it started to actually become narrower beyond 
the year 2009 in absolute terms. 

This is a combination of the empirical 
reduction in standard deviation population as you move 
from 20 years out to 25 years out and the scaling in 
the sense that the demographic forecast also loses 
about 10 per cent of its growth rate in the 2010 to 
2015 period. 

So that effectively, the scaling 
procedure combined with the demographic standard 
deviations resulted in the band narrowing beyond 2009. 
The band for GDP was pretty well constant from about 
2003, 2004 through to 2009, and then it started to 
narrow. And essentially it was my judgment that it 
should not do so. It was not my intent that it do so, 
so I made the change. 

Q. You don't use a different scaling 
factor for each year; do you? 


A. The procedure for scaling we have set 


and for all the years up to 2009 it produced reasonable 


results. But beyond 2009 -- it is not just the scaling 
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but also whether one believes that the standard 
deviation of population really is going to pinch in as 
much as the history would suggest it does. 

Q. So, up to 2009 everything is fine; 
right? 

A. Well, up to 2009, the GDP band 
Satisfies our reasonableness checks. One cannot be 
precise about what the GDP band should be exactly. 
Clearly it exceeds the survey by an amount, it varies, 
but it seems reasonable up to 2009. 

Q. And then in the year 2009 suddenly it 
falls off the table? Suddenly it's wrong. 

A. No. It increasingly falls off the 
table between 2009 and 2015. 

Q. But do I understand it's correct up 
to 2009? You are happy. 

A. I am satisfied that it is reasonable 
to 2009. 

Q. But it is the same equation, isn't 
it, before 2009 and after? 

A. The equation is the same, yes, but 
the equation is not what is determining the -- I mean 
it determines the translation of the population 
Standard deviation into the GDP standard deviation 


effectively, but it is the standard deviation 
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population itself that matters most in all of this 
equation. 

THE CHAIRMAN: I think, Mr. Shepherd - 
excuse me, Mr. Burke - Mr. Burke has given this 
explanation about three times. I mean, you may not 
accept it, it may not be right, but I think he has gone 
over it at least three times that I can think of, this 
particular explanation of why he did what he did and 
all the matters he just spent the last five minutes on. 

MR. SHEPHERD: Mr. Chairman, I was 
actually driving at a different point, although I 
understand I am covering previous ground. 

THE CHAIRMAN: All right. 


MR. SHEPHERD: Q. Mr. Burke, when you 


use an equation to translate one variable into another, 


the equation is the same regardless of where on the 
line you are; isn't it? The equation doesn't change 
over time. It is still the same functional 
relationship between the two variables? 

MR. BURKE: A. Well, it's the same 
equation but the property of both of the equations is 
that the elasticity changes over time; that is, what a 
proportionate change in one explanatory variable does 
to a proportionate change in the dependent variable 


changes. It's a function of the scale of GDP or the 
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scale of population. 

Q. Let me drive at this a different way. 
When you have an equation that explains load in terms 
of population for example -- 

Ae Which WZ «dont. 

Q. Well, load in terms of GDP, which you 
Boe sh git 2 

A. Yes. 

Q. Whatever your GDP number, the 
mathematical relationship between load and GDP is 
identical; isn't that correct? 

A. I guess maybe I am not quite sure 
what you mean by the mathematical relationship. 

Q. When you have an equation load equals 
x times GDP - I am oversimplifying - but let's say x 
times GDP, whenever you change GDP, that produces a 
mathematically identical change in load no matter what 
GDP is; correct? 

A. Well, that is a property of a linear 
System but it is not a property of a quadratic system. 

Q. So your equation changes then as the 
GDP number changes? 

A. The co-efficient is the same, but 
because GDP is there twice, once as a linear term and 


once as a term of degree 2, the effect is that the 
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relationship between GDP and load changes over time as 
specified by the equation. But it's not the same 
Lelacionenip toc -— cthae 1s, just. the polnc@or the 
equation. The relationship does change over time, it 
is not the same. 

Maybe I can help here. I believe that 
the elasticity in the load equation is something around 
1 or slightly higher at the beginning of the period and 
it drops to about 0.85 or slightly less by 2010, 2015, 
so that a given value of GDP does not produce the same 
load as GDP changes over time. 

And that is a property of a forecast as 
well. The median forecast that we arrive at with the 
end use econometric models does not have the property 
that a given value of GDP will necessarily produce the 
same value of load right throughout the forecast 
period. 

OO. Ivam going to ask*you to look “at 
graph 2 of Exhibit 130. Let's just take for argument's 
sake a population of 20,000. 

RLSRY ou mean 20—m1 Livon? 

Q. 20-million, sorry. And compare that 
to the spopulation of 12-million.~ Now, am I correct in 
understanding that your model describes the shape of 


this line exactly the same way at 20-million as it does 
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abi b2-m7 Lilvon? 

A. I have to admit I do not understand 
your question. 

Q. Doesn't a functional equation 
describe the shape of the line? 

A. We have two equations in this graph: 
the population equation and the load equatkions=4 sorry) 
the GDP equation and the load equation, and they are 
strung together to produce this graph. 

Q. I have to admit I am lost 
sufficiently in the math that I can't pursue this any 
further. Let me go back to another area that I think 
maybe I will have an easier time with. 

I am going to show you, if I can find it, 
I think its page 22, the last page of the overheads. 
You will be pleased to know that I have skipped over 
all of those other overheads and won't be coming back 
to them. 

THE CHAIRMAN: I beg your pardon. You 
won't be coming back to them? 

MR. SHEPHERD: I will not be coming back 
to them. 

THE CHAIRMAN: That just raises a 
procedural -- should it then remain as part of the 


exhibit or should it be removed from the exhibit? 
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MR. SHEPHERD: I am happy to take them 
out. It doesn't matter. 

THE CHAIRMAN: Just as a matter of form, 
I guess that we haven't been precisely policing all the 
overheads, but I think most of them have been used up 
to now. But if you are not going to use them, then I 
Suppose they shouldn't be there. 

MR. SHEPHERD: They are -- 

THE CHAIRMAN: Or maybe they should be. 
I have no strong views one way or the other. 

MR.» SHEPHERD: «No, Mr. (Chairman, I> think 
I agree with you. They are numbers 13 through 21, the 
ones that I have not used. 

THE CHAIRMAN: I guess they should be 
just deleted. So that will be pages 13 through 21 of 
Exhibit 126 will be removed. 

THE REGISTRAR: Yes, Mr. Chairman. 

MR. SHEPHERD: I was planning to go 
through the math in detail but I now realize that I 
will never understand my own questions. 

Q. Now, when we are talking about 
uncertainty, we are not talking just about sort of a 
Single concept, are we? There is a whole bunch of 
different types of uncertainty that arise in 


forecasting; isn't that correct? 
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MR. BURKE: A. I think there are 
different sources of uncertainty. 

QvaeuYes ; 

A. I don't know I would use the word 
"types" particularly. 

Q. Okay, sources. 

Now I have prepared a list of ones we 
could think of and I wonder if you could just take a 
look at this list and tell us whether there is anything 
on this list that should not be considered a normal 
source of uncertainty, or, aitennatively,ewaift theres 
anything missing on this list. 

A. Well, the latter point is Certainly 
tall-orderd one -- one moment, please. 
~-~Discussionsoff) the: record: 

MR. BURKE: You have got a variety of 
sources listed here. Typically when people look at 
sources of uncertainty in forecasting, they talk about 
uncertainty 1ingthadatas uncertainty in the 
specification of the model, uncertainty in the driving 
variables or explanatory variables of the model, and 
then a broader class of uncertainty to do with 
Structure of the economy and the context in which the 
whole exercise is being undertaken. 


I think what is on your Page is 
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fundamental changes that may make building an equation 
about the past sort of not relevant for the future that 
it is being used to forecast for. 

And typically one can try to quantify the 
first three kinds. But also one finds oneself saying 
things like we have been saying that some of these are 
fundamental changes in structure, the way the economy 
works, what the rules are, that sort of thing, those 
sorts of things one can't quantify very well. 

Because they haven't happened, they tend 
to be relegated to this class that we have called "low 
probability, high impact" contingencies, which can 
affect the forecasts. And then the problem comes: How 
do you deal with those given that you have said that 


individually they are of low probability? 
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[4:02 p.m. ] MR. SHEPHERD: Q. Let me come back to 
that, okay? I take your point. 

MR. BURKE: A. But looking at the first 
three kinds, I believe that the first five look like 
they refer to the data, roughly speaking, and then 
there are several that refer to how the model is 
specified, and then I guess I am trying to think where 
randomness goes in a source of uncertainty because I 
have a feeling it is not a source of uncertainty, -itiis 
uncertainty. 

THE CHAIRMAN: Perhaps could you explain 
to me what you understand randomness to be? 

MR. BURKE: Well, randomness is a 
property of data that suggests that there is no pattern 
to the data. One talks of a random walk which suggests 
that from one point to the next there is no basis on 
which to predict where the data will move next. No 
model seems to understand it. 

A good example used to be, I am not sure 
whether it still is, the stock market. Individual 
stocks used to be considered a random walk in that it 
was not possible to model how they would move from one 
day to the next, and if you could, then could you make 
a lot money and it wouldn't really work that way in 


practice, everybody would do it, and so on. 
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But it is this property of being without 
structure, without pattern. Is that... 

THE CHAIRMAN: That is fine. Thank you 
you. 

MR. BURKE: I would say I am not sure 
whether economic judgments and political policy -- 
well, economic judgments may refer to No. 8. It might 
refer to model specification, but if it doesn't then I 
would hazard that 8 through to 16 seem to be different 
kinds of structural ways of looking at what I have 
called structural uncertainties. It's uncertainties 
that.are not implicitinytheshistorical data, as’ least 
as we discussed philosophically on Thursday, it's 
probably there somewhere but only by hindsight would 
you realize it, and we are not focusing on it today and 
so we do not pick up some of these changes in the 
forecast. 

MR. SHEPHERD: Q. There is a difference, 
isn't there, between the many judgments you do make in 
forecasting which are sort of judgments within a 
reasonable future, and fundamental change which you say 
you just can't deal with, there is no way of dealing 
whth eit .sn.it that sight? 

MR. BURKE: A. No way of being confident 


quantitatively in the way you deal with it. 
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Q. And there is different types of 
uncertainty depending on whether you are talking about 
fundamental change or you are talking about the 
day-to-day judgments you are making? 

A. Most of the judgments that we have 
been talking, at least that I have been talking about, 
have referred to the first three types or sources of 
uncertainty. 

Q. Yes. You have problems with all of 
these things in uncertainty, in your uncertainty model, 
don't. you; that is to say, all of these types of 
uncertainty one way or another come into your 
forecasting. 

A. Yes. I guess what I am reacting to 
is by giving a whole bunch of different names to the 
same sort of thing, I am not sure whether you aren't 
making it sound bigger than it is. 

I mean, data uncertainty covers a 
multitude of sins and you have got some of them here, 
but'-«the /factidisicthat lyou wean "it absolutely rely on your 
data or you don't have all the data that you want, that 
sont o£ tihangs 

Model specification uncertainties, we 
have discussed the sorts of things that constrain model 


Specification. Model specification can be constrained 
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by the variables for which you have forecasting 
capability. That is, you can only include explanatory 
variables for which you may have a source of forecast, 
and so that may offer a constraint on model 
specification, or for which data is collected. There 
may be variables that you would love to have in an 
equation but no one has collected the data. But it's a 
general class of specification uncertainties. 

Q. So you can have a situation, for 
example, where there simply is no data in an area or 
you can have a situation where you have data but you 
don't think it's reliable data, isn't that correct? 

A. Yes. Telling whether your data is 
reliable or not can sometimes be difficult. 

Q. Wasn't there testimony at one 
point -- I am trying to find the reference. I seem to 
recall Dr. Buja-Bijunas saying at some point that there 
was data for a particular thing for eight years, that 
when you looked at it it didn't look like it was good 
data, not well collected, so you didn't use it. 

A. Well, it may have been myself 
referring to commercial natural gas data, that the 
feedback from the gaS companies in Ontario was that the 
Statistics Canada data for natural gas consumption in 


the commercial sector did not make sense to them. So 
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there is one way of telling whether data is reliable. 
But it comes in that case to having an alternative 
source of information, and quite often you don't. 

Q. Now, let me deal with a couple of 
these, just as examples. Look at underlying 
correlations in the data. That's, for example, 
elasticities; right? You %could™gét your’ elastrcities 
wrong or you could have some uncertainty about the 
elasticities between individual variables? 

A. Well, I don't know what the intent of 
the: word? us*“under lying" is’ here “Ifthe intent is 
that there are correlations you haven't captured, then 
it's not a question of getting elasticities wrong; you 
just haven't specified the model to capture that 
correlation. 

If you are saying you did specify the 
model correctly but then there is uncertainty in the 
estimation of that correlation, certainly that's 
COEnrect: 

Q. Either way it's a particular type 
of -- there is an uncertainty there, a potential for 
error there? 

A. Yes. And I would add that in our 
uncertainty methodology, that sort of uncertainty is 


captured by the standard error on the coefficients of 
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the variables in each equation. 

Q. Now, let me just look at an example. 
Somebody - I don't remember who it was - filed Exhibit 
112, which was your 1987 estimate of residential 
own-price elasticities, and as I recall, it was minus 
.99% 1S that rprighte 

AwgeMy, crecollectionpis. that capplifed-to 
the residential sector. 

0.) Yes.) Andivam Icorrect! in 
understanding that in 1989 you changed that to minus 
38 2 

A. Yes. 

Q. And am I also correct that you now 
think its minus .51? 

A. Yes. And I think I explained to 
Energy Probe, I believe, why we thought the first major 
change had occurred. In fact, both changes are largely 
due to -- because I think there was in each year one 
time frame removed from each of the two equations in 
the residential sector, and the effect of time trends 
was to distort the elasticities, and it's not the sort 
of thing I would hope continue to happen; that is, we 
have improved the estimate of the equations for the 
residential sector and I would hope it's a fairly 


stable specification given the data that we have 
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available. 

If new data sets become available, we 
might be more adventurous in specifications, but given 
the data set which is available, the major cause of the 
change; that is, having time trends in the equations 
and then taking them out sequentially as it turned out, 
explains in my mind why the elasticity changes, and I 
feel that the value we have now is more indicative of 
the correct price elasticity in that previously other 
non-price effects were loaded on to the price variable. 

Q. Now, of course hindsight is 20/20, 
but I guess it's fair to say then that your minus .95 
number in '87, now we see from hindsight had a lot more 
uncertainty in it than your current number. 

A. Well, we are not comparing apples and 
apples here. They are two different equations and that 
equation had a time trend in it. The forecast from 
these two equations was not perhaps as different as the 
change in elasticity might imply. 

Effectively, the time trend was 
offsetting the elasticity and so the forecasts really 
didn't differ that much. 

Q. It's like a double error? 

A. Well, it comes to why we have a time 


trend in the equation in the first place. 
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Q. So, had you estimated a probability 
distribution of that minus .95 number, Say, you might 
have had the same sort of bandwidth on that that you 
would if you estimated today on minus .51? Do I 
understand you to be saying that? 

A. Well, I am not quite sure what we are 
trying to get a bandwidth for. If we are trying to 
Say, what was the standard error of that coefficient-- 

Q. Yes. 

A. --I don't know how they would compare 
in the two years. But I presume both were Significant 
and therefore -- in fact, in this system there is two 
coefficients that didn't need to be looked at to get 
the price elasticity. 

Ont ts? itera’ factor an your answer that we 
know today,or we think we know today, that the minus 
-95 was pretty wildly wrong? 

A. Well, okay, if the issue is what is 
the price response of the equation, then you might say 
it's wrong. 

The issue for us at the time was getting 
the equation to forecast reasonably, and we had to deal 
with an issue about the price forecasts that we were 
working with at the time and we introduced the time 


trend into the equation to help us compensate for what 
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we believed were unrealistic price forecasts at the 
time. 

It is not, from my perspective, a clean 
econometric alternative. It is certainly not the case 
that if we used the same specification today two years 
ago that we would get anything like .95. The data has 
not changed that much in two years. What has changed 
is our forecasting objective, which was to produce an 
equation several years ago that allowed us to not have 
what we believe was unrealistically high increases in 
residential load associated with price changes that 
were then forecasted at the time, and in that sense it 
was quite a manipulative exercise that underlaid that 
equation several years ago. 

We don't have the time trends in the 
model now but I wouldn't want to draw inferences what 
the uncertainty was from a purely statistical point of 
view and how that's change in two years. There is much 
more going in the case of that change in elasticity. 

Q. Let me deal then with another of 
these areas, the lack of data in relevant areas. You 
testified I guess a number of times - "you" 
collectively you - that there are some areas where you 
just don't have data. That's a problem that you have 


to deal with fairly regularly; right? 
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AS’ I YESs 

Q. What do you do? 

A. Well, certainly in the econometric 
modelling exercise, econometricians are in search of 
proxies, things that might have the same statistical 
properties as the missing data that one can find and 
can use, that sort of thing. 

Dr. Buja-Bijunas may have some comments 
as to what happens in end-use modelling when data is 
missing. 

Q. Just let me take an example. You 
testified that you don't have sufficient information on 
the impact of government programs on load to be able to 


relatesmtheitwo; Gisnimttthat sright? 
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[42 D6? Dench] A. Well, it's not clear one would ever 
be able to define an economic variable to represent the 
impact of government programs. 

I mean, the sorts of data we're dealing 
with would be things like expenditures on a particular 
kind of program and hoping to relate those sorts of 
expenditures to an impact on load. We could try that 
at some point in the future when that sort of data 
becomes available, but many programs perhaps don't lend 
themselves to that. 

QO. <I" mynot (being ‘critical. 

A. Like standards, for instance. If 
that's a government program, then there may not be any 
dollars expended by the government to tag that to. So 
you have to use other ways of estimating impact. 
That's one of the reasons that end-use models are so 
important for forecasting because they can look at the 
details and be used to assess implications of specific 
standards or programs. You wouldn't really try to do 
that econometrically. 

Oe Just “to follow that exampleasif, Yin 
fact, government programs did have a very high 
correlation to load - just hypothesize that for a 
moment - and you had no way of determining that 


correlation from data, then absent another variable 
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that proxies government programs, you've created a type 
of uncertainty, haven't you, in which your load 
forecast is responding to something that you haven't 
considered? 

A. In an econometric forecast, one has 
to consider whether all of the implicit trends in the 
variables that one is using as explanatory variables 
are expected to continue into the future or not. If 
you stick with your econometric specification and you 
don't judgmentally adjust the forecast that comes out, 
then, effectively, you are saying all of the things 
that are implicit that may have had the same growth 
pattern over the years as the explanatory variables, 
will continue to have that pattern in future. 

That's one of the reasons why having 
Simpler models ten or fifteen years ago may have led to 
different price elasticities than we get today, that 
implicit in the estimate of coefficient of price - and 
also GDP, for that matter - were things that were 
changing other than, strictly speaking, GDP and price, 
which happened to have the same sort of patterns 
historically as GDP and price. 

But then they started to change, and 
those changes have caused the forecast not to be 


correct. And as a result, people have tended to 
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disaggregate forecasts more and more to break out the 
components. 

And that has led to perhaps a different 
impact on these coefficients. There are other things 
in the equations, other explanatory variables, being 
considered than ten or fifteen years ago. 

Q. Coming back to the government 
programs examples, I take it that that's one of the 
advantages of end-use models then is that they will 
tend to pick that sort of thing up that you can't 
project econometrically? 

A. Well, I wouldn't say they tend to 
Dick vit up; 

Q. That type of model. 

A. The model is able to represent those 
judgments explicitly, whereas it is very difficult to 
stick them into an econometric forecast. 

Q. But the only way it's different is, 
that, in fact, you make a positive judgment tht you'll 
include some sort of factor for a program or a 
standard? 

A. Yes, I think you have to make a 
positive judgment. And that's why we've said when we 
don't have specific information, it's very difficult to 


make a positive judgment as to what the precise impact 
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will be. 

Q. You have to do something, but you 
have not much to work on? 

A. Yes. 

Q. You also have a source of uncertainty 
if there's a relevant factor or some driving variable 
underlying the data that you haven't identified; you 
just don't know it's causing a result. Is that right? 

Let me give you an example. Suppose as a 
matter of fact there's a direct correlation between 
load and frequency of billing, and you just don't know 
it's there. It's somewhere underneath the data and you 
don't know whether it's there. If you miss that, 
that's going to tend to cause error in your forecast, 
LENG ei tio 

A. Well, two steps here. It would cause 
an error in the forecast if the trend in the frequency 
of billing changed. That is, we are modelling all of 
the variation in load with a series of explanatory 
variables, and one way or another it's being captured. 
The question is: Is it reasonable to expect that the 
trend in that particular variable will change at some 
point dramatically? In which case that's when it will 
show that it was part of the cause of a particular 


trend and also then, as it changes, will cause a change 
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in trend that is not explicitly in the model. 

That's what I call a specification error, 
my second kind of error. That is, the model doesn't 
have all or the correct variables, explanatory 
variables, in it. And that may be limited, as I said 
earlier, by the data that is available. 

Q. So, let's just take that example one 
step further. If, in fact, as a matter of human 
behavior it is true that a change to, let's say, weekly 
billing would produce a specific reduction in load and 
you haven't identified that, then that creates an 
uncertainty a your forecast, doesn't it? 

A. Yes, it does. An unquantifiable one. 

Q. An unquantifiable one. You were 
talking about randomness a minute ago. When you 
testified as to load factor, you said one of your 
problems is that there's no discernible pattern in the 
pasto-datas.surlsmthatr true? 

A. What I said was from the early '70s 
to today, it seemed to be a pretty random series for 
the load factor, yes. 

Q. So in that circumstance you just have 
to pick a number; right? You have -- 

A. No. You have a measure of its 


randomness. And so, essentially, in going from an 
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energy band to a peak band, we're taking the product of 
two distributions, and you can estimate what the 
standard error of the load factor is historically and 
Ehen-=- 

You have the energy distribution, which 
we've derived using this procedure we've been talking 
about, and then you take the product of them, because 
the peak is the product of load factor and energy. And 


that ’sj,onevof the things thatisn't intuitively 


. particularly easy to understand because people are used 


to thinking of adding uncertainties -- the variance of 
the sum of two things being the sum of the variance 
minus LweecueHe co-variance. 

But the only way you can estimate the 
product of two things is by simulation, and quite often 
the results you get are not intuitively what you expect 
atival lL. 

And, in fact, when we've done that, we 
get the surprising result that the peak band is 
Narrower than the energy band. Other utilities have 
found that, but we didn't wish to choose to do that 
until we were much more confident in that sort of 
result. 

So, effectively, we have kept the peak 


band and the same degree of uncertainty as the energy 
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band, pending further analysis. 

Q. Mr. Burke, either I'm totally lost, 
which is quite possible, or you haven't quite answered 
the question I asked. Let me go back to the question 
and see whether I can get it. 

As I understand your description, you're 
Saying you can measure the uncertainty around your load 
factor projection; berght? sBubc=sitetime fuiisht- But 
what I'm asking about is how do you get the load factor 
projection? And with no trend in past data, isn't it 
right that you just said, "Let's make a guess and say 
LU scconstant;* 2 

A. No. That's not a guess. That's the 
best forecast, given a series which is fluctuating at 
random about a constant mean. And we've checked that 
it does statistically fluctuate about a constant mean. 

Thatlsrinot a> gquesssrn That "sa--syourk now, 
some things have trends; some things stay constant. 
That's the best forecast for the load factor. 

Q. So when you said that the past data 
doesn't have any discernable pattern -- you've said 
that number of times in your past -- 

A. Doesn't have any discernable trend, I 
think I said. Well, okay. 


Q. 79 Okay. 
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Aa BOEhMofsethempare trueje2dt isla 
random process around a constant mean for the last 20 
years. 

Q. Okay. Maybe I'm just going to skip 
over to what is, after all, the point of all of this. 
Your particular bandwidth methodology, the particular 
way that you choose to describe uncertainty, can you 
just give us an idea of which of these it picks up 
totally and which ones it doesn't pick up, and which 
ones it sort of partly picks up? 

A. Pretty well the data issues. It's a 
difficult question. I'll have - think) aboutsithforta 
minute because this isn't particularly the way I would 
categorize uncertainty. 

I think probably the difficulty we're 
having answering may relate to this hybrid approach. 
That is, a lot of the issues associated with the data 
reliability, the specification and so on, are captured 
in the production of the forecast using the econometric 
and end-use approaches. And to the extent that the 
median forecast is wrong because of data problems or 
absence of data, poor specifications and so on, that is 
reflected in the general trajectory of the median 
forecast. 


And we had a single equation that fits 
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that. forecast) atrackeiatiwelljPand its £10 -—— in’ trying 
with just a single equation to fit total load, I think 
that is why the residual standard error may be fairly 
large, compared to some other models, and the 
co-efficient standard errors, well, they're not too 
large, I have to admit. That may be where, 
essentially, the extent to which those kinds of errors 


are reflected in the uncertainty band. 
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[4.38 p.m. ] Dathinkiqhnattstright! I think that the 
data uncertainties, the model specification 
uncertainties, are captured in the extent to which one 
can track the forecast with the single equation and 
therefore are implicit in the residual standard error 
of the equation and in the coefficient error of this 
equation. 

Qugekt tsi ifars too rcomplicated forme. In 
simple terms, does that mean that your uncertainty 
bandwidth captures those things, a hundred per cent 
captures those things? 

A Tocthertextentrrhat Siti si possib1e ito 
quantify the impact of data uncertainty, specification 
uncertainty, in the aggregate for total system 
electricity demand, I believe the equation captures 
those things, but does not capture structural changes 
such as the remaining 8 to 16. 

Q. So it doesn't capture the level of 
reliability of your judgments? Like your economic 
judgments, Kuwait will need more capital, Eastern 
Europe will need more capital, things like that? It 
doesn't capture the uncertainty associated with whether 
those judgments are right, or it does? 

A. Oh, looking at economic judgments. 


Is that what you meant by economic judgment? 
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Q. For example, yes. 

A. There are very many judgments in the 
forecasts; and to a certain extent, a lot of them may 
be offsetting. Indeed, each of the detail judgments, 
it is not clear how any individual judgments 
contributes to uncertainty about the forecast. I think 
that's the problem of producing a band for the total 
system. It is looking at the aggregate effects of all 
of these elements that go into the forecast. 

Q. I understand the problem but there is 
an uncertainty about whether any given judgment is 
correct, yes? Or how correct it is going to be? 

A. Individually, there is an uncertainty 
about each and every judgment of the thousands that 
must underlie the median forecast. 

Q. Let me get at this in my plodding 
way. Is it intended that your bandwidth capture all of 
those uncertainties in aggregate a hundred per cent? 

A. It is the intent that the 
relationship between GDP and load either through the 
uncertainty in GDP itself or the uncertainty in the 
relationship between GDP and load captures all of these 
factors, yes. 

Qs6 So,pthesonly thing that isn’t 


captured is fundamental change, the structural change? 


Farr & Associates Reporting, Inc. 


24 


25 


Rothman, Burke,Buja-Bijunas 2185 
cr ex (Shepherd) 


Bai Yess 

Q. Am I right in assuming that because 
you have an 80 per cent confidence band, that implicit 
in that is an assumption that there is less than 20 per 
cent likelihood of non-offsetting structural change? 

A. I think that's the judgment we must 
be making at this point in setting the forecast where 
it is set, yes. 

Q. Does that mean that there is sort of 
an additional uncertainty associated with anything like 
this beyond your bandwidth technique? 

Altnthere isravctass tof sinceertainty that, 
as I have said, is unquantifiable. That makes it very 
difficult to know where it is. And there is nothing 
that one is going to do to eliminate that. 

Q. I am not suggesting you are not doing 
what you should be doing. I guess I am trying to 
understand when you say the bandwidth now 17,900 
megawatts, that 17,900 megawatts plus some additional 
uncertainty which can't been quantified. 

A. Well, this is the 80 per cent band. 
You could have a 99 per cent band and get a much wider 
band. 

Q. That is not what I am asking you. 


A, hewWellgatithinkhet 2s,vactually, 
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because as the confidence level increases, more and 
more of these contingencies can be said to be included. 

By 2015 what seems like an extremely low 
probability event today may be quite important, and we 
would dismiss it entirely at this point in time, but it 
may be what turns out to determine load and it's 
probably assigned a half a per cent probability by 
anybody you ask their opinion about it today. 

For instance, if you ask someone: What 
is the change that we will be in depression in Ontario 
for the next 25 years in a row? They will probably 
say, “Almost no chance at all." It could happen. 
Extremely unlikely. How much weight should be put ona 
scenario like that? A tenth of a per cent, half a per 
cent?) Dudon: t iknow.) sThattis: what's ¢@i FEicurts*©*But 
that is sort of is one way of getting an extremely low 
result that would over 25 years put you way below the 
low end of our band, but what is the probability of 
that particular scenario. 

Q. I want to come to the final point, 
hopefully. With all of these uncertainties associated 
with forecasting, the average lay person might say why 
bother do it? What is it you learn from it? And even 
the bandwidth you have projected is pretty wide; isn't 


ite? 
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A. I think the planners are having 
trouble keeping up with it. 

Ongiisngt dace faiesttotsayfthatbtheterare 
other things you learn from going through the process, 
the techniques you use teach you a lot of other things 
about interrelationships between variables and factors 
in the economy, and that those are also a valuable 
aspect of the process? 

A. Of modelling uncertainty? 

Q. Of modelling load or modelling GDP? 

Az 20h. ),Webbpetitstnofwalbynitherevis a 
premise here, I think, that you were sort of were 
Saying something about why bother doing it because it's 
all so uncertain. Am I supposed to agree to that? 

Q. No, I was saying one might ask that 
question, yes. 

A. I'm sorry, are you asking the 
question? 

Q. Okay, I am asking the question. 

A. Pardon? 

Q. I am asking the question. What's the 
point if you have a wide uncertainty band? 

A. Because it is the best information 
you have for planning purposes. And I think if we all 


agree not to bother planning, then we can abandon this 
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exercise. But essentially in order to plan, you need a 
good sense of where you are going in the future and it 
doesn't rule out that you can change your future, but 
you do need to know where you would be if you didn't 
take major steps. That's what the basic load forecast 
is about and that's what we have attempted to do. I 
think this gets back to one of the first things I said 
in my direct evidence, which is we are trying to make 
informed and balanced forecasts of the future. 

Q. Do I take it then that in your view 
you cannot plan unless you first forecast? 

A. That's correct. And I discussed this 
issue both in direct evidence, and I think various of 
the intervenors were suggesting that targetting in some 
sense was an alternative. Maybe I was reading 
something into their questions. But my position is 
that you cannot target meaningfully without 
forecasting. 

Q. Let me get back to the question I was 
asking. During the course of the forecasting process, 
your techniques teach you a lot about the relationship 
between variables within society and within the 
economy, don't they? 

A 'Yes*% 


Q. And that information is valuable 
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aside from forecasting? 

A. Yes, absolutely. And one of the 
major uses of the end-use system, as we have described, 
is that it provides valuable information for the design 
and later on evaluation of demand management programs. 

Q. Now, it has been suggested that 
instead of starting with a forecast and going to the 
plan, you could start with the plan and figure out what 
happened as a result of doing that, right? It is the 
backcasting approach. 

A. Well, I must say that's not quite my 
didemsdanaias of backcasting. My understanding of 
backcasting is that you set a goal. And I would have 
thought of it in terms of some results like not to have 
any growth in load, for instance. 

Ow Yes. 

A. Now, I guess if you are saying you 
have already decided how you can achieve that goal, 
then you are starting with a plan. I don't believe you 
can meaningfully decide that you know that you can 
achieve that goal until you have really figured out how 
hard it is going to be to achieve, and you really can't 
tell that unless if you have figured out what would 
have happened if you didn't do all of the extra things 


that you are going to do to somehow make zero per cent 
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happen instead of 2 per cent. 

Q. Could the planners come back to you 
and say, we don't really care what happens if we do 
nothing because we know we are going to do something. 
What we would like to know is can you run your models 
and tell us what sorts of things would cause this 
result, this let's say, zero load growth. Could you do 
your models that way? Could you run your models that 
way? 

A. Yes, the econometric model, you would 
be doing some pretty blunt things, it wouldn't be very 
ea But the end-use model you certainly could 
produce theoretically an infinite number of scenarios 
as to how combinations of load and efficiency and 
whatnot, ins the) futareisthat getséyou)avparticul ar 
result. 

The problem in practice is that there's 
an infinite number of ways to do it; whether any of 
them are feasible is yet another question. But if you 
want to know is it possible to add up load to get zero 
growth at a point in time in the future, you can 
certainly do that with an end-use model. 

And that's the reason why when people 
have been asking us about sensitivities and we say 


well, we haven't looked at that one, it's because there 
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is SO many you can look at. You could decide to 
eliminate this market, eliminate that market, do this, 
do that. There are infinite numbers of ways of doing 
that. Are they feasible, practical, acceptable, 
desirable, any of those words. Very hard to tell. And 
until someone makes it seem plausible that any one of 
them is a useful avenue to pursue, it isn't 
particularly useful for us to sort of paint the whole 
pictured! teiss,anhkotsof works 

Q. What I was talking about a minute ago 
is backcasting, right, as you have redefined it. But 
there is also scenario building which is different; 
isnut wit 2 

A. Scenario building is different, yes. 

Q. And how is it different? 

A. Well, a scenario is a complete 
forecast, an alternative forecast perhaps in the way 
the term is usually used, but you can certainly 
describe our median case as a scenario. It is one way 
of going from here to there, so to speak, within 
internally consistent assumptions. We happen to think 
that it is the most likely, the median case, so it's 
Our preferred scenario. 

So there is nothing special about the 


word "scenario," but it does require an internally 
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consistent set of forecast information. And it has 
zero probability in and of itself. 

Q. Am I right in understanding your 
approach to posit a single future, a single line future 
with variations on that future only? 

A. Our load forecast is a single 
scenario which is deemed to be the median, most likely, 
whatever, pretty close in practice, scenario with a 
associated uncertainty. That forecast has considered a 
wide range of contingencies, and those go into the 
formation of the judgments about the specific forecast. 

For instance, there has been a lot of 
discussion with Mr. Rothman about his GDP assumptions, 
and certainly some consideration needs to be given to 
departures from the median case. What sort of things 
could cause departures? But having looked at the range 
of possible outcomes for GDP, one is selected as the 


basis, yes. 
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(A250 pom) MR. ROTHMAN: A. I hesitate to jump in 
here. 

Q. I am almost finished. 

A. That's the only reason why I should 
hesitate. I think it is important to recognize the 
point about the forecast being a scenario, but that I 
think I heard you ask a question that went something 
like, is this a median forecast and there are 
variations around it. 

The point of the uncertainty band is to 
Capture at least some of what would be alternative 
scenarios though not alternative scenarios that are 
very different; that is, when we build a scenario we 
build it up, we build the forecast by making a set of 
assumptions, that's really the discussion that we had 
on Thursday. 

We could build a different forecast with 
a different set of assumptions. Any time you change 
one of those assumptions you will build a different 
forecast. It's important when you change one 
assumption to build a different forecast as a scenario, 
that is, to keep the assumptions together, coherent, so 
that you don't have an assumption from one source here 
and an assumption from another source here, coming from 


very different points of view and Producing assumptions 
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that aren't necessarily compatible and try to put them 
together into a forecast. That's why we do a scenario. 

But I think that where we are talking 
about changes in assumptions that aren't radical 
changes, that really has to be -- not changes in 
assumptions, differences of the actuals from our 
assumptions within the same pattern, in effect, that 
has to be captured in the uncertainty band rather than 
captured by -- rather than produce what would be an 
entire alternative scenario. 

Q. Now, there are other areas in which 
you use scenarios today; right? Non-load forecast 
areas? 

A. Yes 

Q. Didn't you testify that you use it 
for your treasury division? 

A. Yes. 

Q. Do I understand your testimony to be 
that you used to provide uncertainty bandwidth around 
interest rate projections for your treasury division 
and they didn't find that very useful and actually used 
the scenarios instead? 

A. Yes, and for precisely the reason 
that I was talking about. They needed something more 


than just a single variable uncertainty with no notion 
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of what the causal effects were. They needed a 
coherent scenario so that they could put together the 
variables that are relevant to them. 

MR. BURKE: A. I would like to add at 
this point that the reason we are producing an 
uncertainty band is that the users of the load forecast 
information do not find scenarios in themselves without 
some probability that they can assign to them useful 
for planning purposes. 

We did look, as we have discussed earlier 
and which is located in one of the Northwatch 
interrogatories that's already on the recoravy’that 
alternative scenarios as a basis for contingency 
planning but decision-making based on alternative 
scenarios seems to be very difficult in practice. 

Q. And treasury division can do it but 
the planners can't? 

A. Maybe it's the nature of the problem 
they have to solve. 

MR. ROTHMAN: A. Treasury division uses 
it to test their financing strategies against the 
alternative scenarios. 

MR.* BURKE :) Asay) Litemay? bethat 
judgmentally they are much happier with assigning 


weights to scenarios and doing that very judgmental 
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process that is involved when you start to use 
alternative scenarios as a basis for planning. 

But the system planners who have to weigh 
decisions and risks to the plan associated with high 
and low or changing or whatever patterns of load, find 
it not feasible to do that with a series of scenarios 
which no one can tell them the likelihood of. They 
don't wish to assign probabilities to the scenarios; 
they want someone to assign probabilities for them. 

And the only way we can really assign probabilities to 
future load information, which is really what the range 
forecast is, is through a range or a probability 
distribution. 

Q. Maybe I am misunderstanding this. Is 
the reason why you use scenarios in one area and 
bandwidth methodology in the other area simply that the 
end users in each case have a preference for one or the 
other, or is there some fundamental difference between 
planning financing strategy and planning building 
strategy? 

A. I don't know that I want to speak for 
the treasury people, but I think that it is fair to 
observe that the nature of the planning information is 
probably a function of what is being planned, how 


useful it is to arrive in certain forms or others, and 
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where the judgments are and are not made. 

In practice, for load forecasting 
purposes, when we produced contingency scenarios that 
had almost no probability of occurring but certainly 
bounded the likely futures, that information was almost 
of no value to system planners. They did not know how 
to plan under-uncertainty given that sort of 
information, and they really needed some way to weight 
the information in a meaningful sense. 

And given that each individual scenario 
did not claim to cover the universe, that is, you could 
not provide a set of scenarios which, when added up, 
covered all of the universe. The only way you can do 
that is to come up with some very boring scenarios that 
each have a sort of a wedge into the future and cover 
the same thing as an uncertainty band. 

In effect, that is what in the end we are 
doing. We are using the uncertainty band to provide 
probabilities on extreme outcomes without having to 
specify exactly which set of events causes that 
Outcome, and there are many ways to have extreme 
outcomes when you are dealing with something as diverse 
as electricity demand. So it almost doesn't matter how 
you get that outcome; it matters what is the chance 


that you will end up in that extreme situation. 
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Q. I take it from the fact that you use 
both methods, depending on what the users ask for, that 
you don't have a particular preference for using a 
bandwidth methodology or using scenario generation 
methodology? 

A. We used to produce both. And for the 
load forecast, the demand for scenarios dried up and 
it's a lot of work so we stopped doing it. 

Q. If the planners came back to you then 
and said, we have got all these wonderful new models 
and we would like you to run some scenarios, let's say, 
Keb randud. and) Ay BeandsG, your techn rques*courd do 
that fine, couldn't they? 

A. Yes. 

Q. So if they said to you, do a scenario 
in which we, Ontario Hydro, are limited to demand side 
management and non-utility generation, we have no other 
generation options, could you do that? 

A. Well now, I need some information in 
additions tovjustithat 2° I- need? to know!=— the~ price of 
electricity and, for that matter, the price of natural 
gas would start to be important. 

Q. But they could specify that and you 
could run that scenario? 


ABPUITE*Chey-specirfyear. of that’ Sture; 
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certainly the structures are there to analyze, again 
given all the caveats we made about those prices not 
moving out of the range for which we have comfort in 
the elasticities; as soon as that happens, we would 
then have to do what amounts to a micro analysis of 
what decisions people would make under extreme 
Circumstances. 

Q. Even if you did that scenario, there 
would still be uncertainty around that scenario, 
wouldn't there? 

A. As we have said repeatedly, there is 
absolutely no certainty with any scenario. The 
probability of any individual scenario is zero. You 
have to define a region around it somehow in order to 
assign probability to it, and that's a Pretty arbitrary 
exercise. 

Q. My last question is, the planners 
haven't actually asked you to do any of that, produce 
that sort of scenario or any scenarios? 

A. They used to. But now since about 
1988 they have been interested in a probability 
distribution about the load forecast. In my view that 
represents an evolution from scenarios to uncertainty 
bands; that is, it was very popular to look at 


scenarios in the early '80s and it certainly does 
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sharpen a corporation to consider the contingencies 
with which it could be faced. 

Having done that, though, you begin to 
learn what it is you can and cannot react to, and where 
flexibility, which is the main outcome of these sorts 
of exercises, can lead to you. And having addressed 
where is it that you can buy flexibility, if you really 
want to make the trade-offs as to how much flexibility, 
you need to be able to assign weight to those chances, 
and that's where uncertainty bands come in. So I would 
Say they come after a consideration of scenarios. 

MR. SHEPHERD: Mr. Chairman, I have no 
further questions. 

THE CHAIRMAN: All right; we will stop 
for the day. 

I'm sorry, Mr. Thompson, but even if you 
do take five minutes, that would be five minutes too 
long. We have to stop at five o'clock sharp comignt:. 

THE REGISTRAR: This hearing will adjourn 
until tomorrow morning at ten o'clock. 

---Whereupon the hearing was adjourned at 5:00 p.m., 


to be resumed on Tuesday, May 14, 1991, at 10:00 
a.m. 
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